Vouch Server Operator Guide

This documentation covers deploying, configuring, and operating Vouch Server — the authentication server that issues short-lived credentials after FIDO2 verification with a YubiKey. It covers three deployment patterns: cloud, on-premise, and air-gapped.

For CLI installation, enrollment, integration guides (SSH, AWS, EKS, GitHub, Docker), and OIDC provider documentation (endpoints, tokens, grant types), visit vouch.sh/docs.

What Vouch Server Does

Vouch Server is the backend that makes hardware-backed authentication work:

• OIDC Provider — Issues DPoP-bound access tokens after FIDO2 verification
• SSH Certificate Authority — Signs short-lived Ed25519 certificates
• Credential Broker — Exchanges access tokens for AWS STS credentials
• SCIM Endpoint — Receives user provisioning/de-provisioning from your IdP
• WebAuthn Relying Party — Manages FIDO2 credential registration and assertion

Architecture

Security

Vouch is designed for high-security environments:

• Memory-safe implementation — Written in Rust
• No credential storage — Vouch never sees your private keys
• Cryptographic presence attestation — FIDO2 with user verification
• Short-lived credentials — Minimize blast radius of compromise
• Audit trail — Every credential issuance logged with attestation

Get started with the Deployment Overview.

Deployment Overview

This section covers deploying the Vouch server for your organization. The server is the central authentication backend that handles FIDO2 verification, session management, SSH certificate signing, and OIDC token issuance.

Deployment Checklist

Before deploying, ensure you have:

• Domain name — A domain for your Vouch server (e.g., auth.example.com)
• TLS certificate — Valid certificate for your domain (or use Let’s Encrypt)
• Database — SQLite (single node) or PostgreSQL (multi-node)
• Identity provider — Google Workspace
• JWT secret — Cryptographically random string, minimum 32 characters (or use AWS KMS HMAC)
• SSH CA key (optional) — Ed25519 key pair for signing SSH certificates (or use AWS KMS)
• OIDC signing key (optional) — P-256 EC key for signing ID tokens (or use AWS KMS)

Architecture

                    Internet
                       │
                       ▼
              ┌─────────────────┐
              │  Load Balancer  │
              │  (TLS termination │
              │   or passthrough) │
              └────────┬────────┘
                       │
                       ▼
              ┌─────────────────┐
              │  Vouch Server   │
              │                 │
              │  • Auth Portal  │
              │  • OIDC Provider│
              │  • SSH CA       │
              │  • REST API     │
              └────────┬────────┘
                       │
                       ▼
              ┌─────────────────┐
              │    Database     │
              │                 │
              │  SQLite or      │
              │  PostgreSQL     │
              └─────────────────┘

Deployment Methods

Configuration

All configuration is via environment variables. See the Configuration Reference for the full list.

The minimum configuration requires:

VOUCH_RP_ID=auth.example.com        # Your domain
VOUCH_JWT_SECRET=<64-char-secret>    # Session signing secret
VOUCH_DATABASE_URL=sqlite:vouch.db?mode=rwc  # Database

For production, you’ll also want:

VOUCH_TLS_CERT=<base64-encoded-pem>  # TLS certificate
VOUCH_TLS_KEY=<base64-encoded-pem>   # TLS private key
VOUCH_SSH_CA_KEY=<base64-encoded-pem> # SSH CA key (or VOUCH_SSH_CA_KMS_KEY_ID)
VOUCH_IDPS=google                                    # External IdP(s)
VOUCH_IDP_GOOGLE_TYPE=oidc
VOUCH_IDP_GOOGLE_ISSUER=https://accounts.google.com
VOUCH_IDP_GOOGLE_CLIENT_ID=...
VOUCH_IDP_GOOGLE_CLIENT_SECRET=...

For AWS deployments, you can use KMS for all signing operations instead of managing local keys. See the Configuration Reference for KMS options.

Sizing

The server is single-process, async (tokio). Per-session memory overhead is minimal (~2 KB for token metadata). The primary bottleneck is database I/O during token issuance and session validation.

Database guidance:

• SQLite — suitable for single-node deployments under ~500 users
• PostgreSQL — recommended for multi-node or >500 users
• Aurora DSQL — for AWS deployments requiring managed infrastructure

Next Steps

1. Database Setup — Choose and configure your database
2. TLS Configuration — Set up HTTPS
3. Configuration Reference — Full environment variable reference
4. Identity Provider Setup — Connect your corporate IdP

Configuration Reference

This chapter describes the Vouch server configuration system, including configuration sources, S3-based configuration, and hot-reload behavior.

Vouch server supports multiple configuration methods with a defined precedence order.

Configuration Sources (Priority Order)

1. S3 Configuration (highest) — JSON file fetched from S3
2. Environment Variables — Standard VOUCH_* prefixed variables
3. Command-line Arguments — Direct CLI arguments

When S3 configuration is enabled, it overrides environment variables. This allows for centralized configuration management with dynamic updates.

S3-Based Configuration

For production deployments, Vouch supports loading configuration from an S3 object. This enables:

• Centralized management — Single source of truth for multi-instance deployments
• Dynamic updates — Configuration changes without server restart (for supported fields)
• TLS hot-reload — Automatic certificate rotation without downtime
• Secrets management — Leverage S3 encryption and IAM for credential protection

Enabling S3 Configuration:

# Required: bucket name
VOUCH_S3_CONFIG_BUCKET=my-bucket

# Optional: object key (default: config/vouch-server.json)
VOUCH_S3_CONFIG_KEY=config/vouch-server.json

# Optional: AWS region (uses default credential chain region if not set)
VOUCH_S3_CONFIG_REGION=us-west-2

# Optional: polling interval in seconds (default: 60)
VOUCH_S3_CONFIG_POLL_INTERVAL=60

S3 Configuration JSON Schema:

{
  "version": 1,
  "listen_addr": "0.0.0.0:443",
  "rp_id": "vouch.example.com",
  "rp_name": "Example Corp",
  "base_url": "https://vouch.example.com",
  "database_url": "postgres://...",
  "jwt_secret": "32+ character secret",
  "tls": {
    "cert": "<base64-encoded PEM certificate>",
    "key": "<base64-encoded PEM private key>"
  },
  "idps": [
    {
      "id": "google",
      "type": "oidc",
      "issuer": "https://accounts.google.com",
      "client_id": "...",
      "client_secret": "..."
    },
    {
      "id": "corp-saml",
      "type": "saml",
      "metadata_url": "https://idp.example.com/saml/metadata",
      "sp_entity_id": "https://vouch.example.com"
    }
  ],
  "allowed_domains": ["example.com"],
  "ssh_ca_key": "<base64-encoded PEM Ed25519 private key>",
  "ssh_ca_kms_key_id": "mrk-...",
  "oidc_signing_key": "<base64-encoded PEM EC P-256 private key>",
  "oidc_signing_kms_key_id": "mrk-...",
  "oidc_rsa_signing_key": "<base64-encoded PEM RSA-3072 private key>",
  "oidc_rsa_signing_kms_key_id": "mrk-...",
  "jwt_hmac_kms_key_id": "mrk-..."
}

See the S3 Configuration Schema for the full field reference.

All certificate and key fields are base64-encoded PEM strings:

# Encode a PEM file for S3 config
base64 -i cert.pem | tr -d '\n'

AWS KMS Signing Keys

As an alternative to managing local key material, Vouch supports AWS KMS for signing operations:

Multi-region keys (mrk- prefix) are recommended for high availability. KMS key IDs can also be set in the S3 config (ssh_ca_kms_key_id, oidc_signing_kms_key_id, jwt_hmac_kms_key_id).

See Key Management for generation and rotation details.

Hot-Reloadable vs Startup-Only Fields

Non-hot-reloadable fields include: jwt_secret, database_url, listen_addr, rp_id, rp_name, session_hours, cors_origins, allowed_domains, dpop.*, OIDC settings, SAML settings, GitHub App settings, SSH CA key, OIDC signing keys, and all KMS key IDs.

Changes to non-hot-reloadable fields in S3 are silently ignored. A server restart is required to apply them.

TLS Certificate Hot-Reload

Vouch supports automatic TLS certificate reloading without dropping connections:

1. Via S3 polling — Update tls.cert and tls.key in S3 config; server detects change via ETag and reloads
2. Via SIGHUP — Send SIGHUP to the server process to reload TLS certificates

# Manual TLS certificate reload (Unix only)
kill -SIGHUP $(pgrep vouch-server)

Note: SIGHUP only reloads TLS certificates. It does not reload any other configuration fields.

Database Setup

Vouch supports three database backends. Choose based on your deployment requirements.

SQLite (Default)

Best for single-node deployments and development. No external dependencies.

VOUCH_DATABASE_URL=sqlite:vouch.db?mode=rwc

The database file is created automatically on first startup. Migrations run automatically.

Recommendations:

• Store the database on a persistent volume
• Set restrictive file permissions: chmod 700 /data
• Back up the file regularly (it’s a single file)

# Create data directory
mkdir -p /data
chmod 700 /data

# Configure
export VOUCH_DATABASE_URL="sqlite:/data/vouch.db?mode=rwc"

PostgreSQL

Best for multi-node deployments, high availability, and production environments.

VOUCH_DATABASE_URL=postgres://user:password@db.example.com:5432/vouch

Setup:

1. Create a PostgreSQL database:

   CREATE DATABASE vouch;
   CREATE USER vouch WITH PASSWORD 'secure-password';
   GRANT ALL PRIVILEGES ON DATABASE vouch TO vouch;
   

2. Configure the connection:

   export VOUCH_DATABASE_URL="postgres://vouch:secure-password@db.example.com:5432/vouch"
   

3. Migrations run automatically on server startup.

Recommendations:

• Use SSL for database connections in production
• Configure connection pooling at the database level
• Set up automated backups

Aurora DSQL

For AWS deployments requiring serverless, distributed SQL with strong consistency.

Aurora DSQL endpoints are auto-detected when the DATABASE_URL hostname contains .dsql. and ends with .on.aws. IAM authentication tokens are generated automatically.

VOUCH_DATABASE_URL=postgres://admin@abcdef123456.dsql.us-east-1.on.aws:5432/vouch

Multi-region configuration uses a dsql_endpoints map in the S3 configuration JSON, resolved via AWS_AZ or AWS_REGION environment variables.

Migrations

Database migrations are embedded in the server binary and run automatically on startup. There is no manual migration step required.

• SQLite migrations: crates/vouch-server/migrations/sqlite/
• PostgreSQL migrations: crates/vouch-server/migrations/postgres/

Backup

Always back up before upgrading the Vouch server, as migrations may modify the schema.

TLS Configuration

Vouch requires HTTPS in production. TLS can be configured directly on the Vouch server or terminated at a load balancer.

Direct TLS (Recommended)

When TLS is configured, the server automatically:

• Listens on port 443 (HTTPS)
• Runs an HTTP redirect server on port 80 (308 redirect to HTTPS)
• Makes the /health endpoint accessible on HTTP (for load balancer health checks)
• Validates the Host header against rp_id to prevent injection attacks
• Ignores VOUCH_LISTEN_ADDR (ports are fixed at 443/80)

Note: Binding to ports 80 and 443 requires CAP_NET_BIND_SERVICE capability on Linux. The RPM/DEB packages configure this automatically.

Configuration

Provide base64-encoded PEM certificates via environment variables:

# Encode your certificate and key
export VOUCH_TLS_CERT="$(base64 -i cert.pem | tr -d '\n')"
export VOUCH_TLS_KEY="$(base64 -i key.pem | tr -d '\n')"

Both VOUCH_TLS_CERT and VOUCH_TLS_KEY must be set together. If only one is provided, the server will fail to start.

TLS Properties

• Protocol: TLS 1.3 and TLS 1.2
• Implementation: rustls (no OpenSSL)
• Ciphers: BCP 195 (RFC 9325) suites only — TLS 1.3 AEAD suites, and ECDHE+AEAD suites for TLS 1.2 (AES-GCM, ChaCha20-Poly1305)

Post-Quantum Key Exchange

Both TLS listeners (HTTPS and mTLS) and all outbound TLS clients (CLI, agent, and server-to-IdP/AWS connections) prefer the X25519MLKEM768 hybrid post-quantum key-exchange group. When the peer supports it — modern browsers, Cloudflare, and AWS endpoints do — the TLS session keys are protected against “harvest now, decrypt later” attacks. Peers without ML-KEM support negotiate classical X25519 or P-256 as usual; no configuration is required on either side.

See Post-Quantum Cryptography for Vouch’s overall PQC posture.

Certificate Hot-Reload

Vouch supports automatic TLS certificate reloading without dropping connections. This is useful for certificate rotation (e.g., Let’s Encrypt renewals).

Via S3 Configuration

If using S3 configuration storage, update the tls.cert and tls.key fields in the S3 config file. The server detects changes via ETag polling and reloads automatically.

Via SIGHUP

Send SIGHUP to the server process to reload TLS certificates:

kill -SIGHUP $(pgrep vouch-server)

Note: SIGHUP only reloads TLS certificates. It does not reload any other configuration.

Self-Signed Certificates (Development)

For development or testing:

# Generate self-signed EC certificate
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
  -keyout tls_key.pem -out tls_cert.pem -days 365 -nodes \
  -subj "/CN=localhost" \
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1"

# Base64 encode for Vouch
export VOUCH_TLS_CERT="$(base64 -i tls_cert.pem | tr -d '\n')"
export VOUCH_TLS_KEY="$(base64 -i tls_key.pem | tr -d '\n')"

Deployment Methods

Choose a deployment method based on your infrastructure:

All methods use the same Vouch server binary and configuration via environment variables. The deployment method only affects how the process is managed and how configuration is provided.

Verification

Regardless of deployment method, verify the server is running:

# Health check
curl -k https://auth.example.com/health
# Expected: {"status":"healthy"}

# SSH CA public key (if configured)
curl -k https://auth.example.com/v1/credentials/ssh/ca
# Expected: ssh-ed25519 AAAA... vouch-ca@...

# OIDC discovery
curl -k https://auth.example.com/.well-known/openid-configuration
# Expected: JSON discovery document

Systemd (Bare Metal)

Deploy Vouch as a systemd service on bare metal servers or VMs.

Install via Package

The RPM and DEB packages include a systemd service unit:

# RPM (RHEL/Fedora/Amazon Linux)
rpm -ivh vouch-server-1.0.0-1.x86_64.rpm

# DEB (Debian/Ubuntu)
dpkg -i vouch-server_1.0.0_amd64.deb

The package installs:

• Binary at /usr/bin/vouch-server
• Systemd unit at /etc/systemd/system/vouch-server.service
• Default config at /etc/vouch/vouch.env
• Data directory at /data (with appropriate permissions)

Configure

Edit the environment file:

sudo cp /etc/vouch/vouch.env /etc/vouch/vouch.env.local
sudo chmod 600 /etc/vouch/vouch.env.local
sudo vi /etc/vouch/vouch.env.local

At minimum, set:

VOUCH_RP_ID=auth.example.com
VOUCH_JWT_SECRET=<your-64-character-secret>
VOUCH_DATABASE_URL=sqlite:/data/vouch.db?mode=rwc
VOUCH_TLS_CERT=<base64-encoded-certificate>
VOUCH_TLS_KEY=<base64-encoded-private-key>

See Configuration Reference for all options.

Start the Service

# Enable and start
sudo systemctl enable --now vouch-server

# Check status
sudo systemctl status vouch-server

# View logs
sudo journalctl -u vouch-server -f

Manual Install (Without Package)

If installing the binary manually:

1. Copy the binary:

   sudo cp vouch-server /usr/bin/
   sudo chmod 755 /usr/bin/vouch-server
   

2. Create a systemd unit:

   # /etc/systemd/system/vouch-server.service
   [Unit]
   Description=Vouch Identity Server
   After=network.target
   
   [Service]
   Type=simple
   User=vouch
   Group=vouch
   EnvironmentFile=/etc/vouch/vouch.env
   ExecStart=/usr/bin/vouch-server
   Restart=on-failure
   RestartSec=5
   
   # Security hardening
   NoNewPrivileges=true
   ProtectSystem=strict
   ProtectHome=true
   ReadWritePaths=/data
   AmbientCapabilities=CAP_NET_BIND_SERVICE
   
   [Install]
   WantedBy=multi-user.target
   

3. Create the service user and directories:

   sudo useradd -r -s /sbin/nologin vouch
   sudo mkdir -p /etc/vouch /data
   sudo chown vouch:vouch /data
   sudo chmod 700 /data
   

4. Reload and start:

   sudo systemctl daemon-reload
   sudo systemctl enable --now vouch-server
   

Upgrading

# Back up database
sudo cp /data/vouch.db /data/vouch.db.backup.$(date +%Y%m%d)

# Upgrade package (migrations run automatically on next startup)
sudo rpm -Uvh vouch-server-1.1.0-1.x86_64.rpm
# or: sudo dpkg -i vouch-server_1.1.0_amd64.deb

# Restart
sudo systemctl restart vouch-server

# Verify
curl -k https://auth.example.com/health

Docker

Deploy Vouch using Docker or Docker Compose.

Docker Run

docker run -d \
  --name vouch-server \
  --restart unless-stopped \
  -p 443:443 \
  -v vouch-data:/data \
  -e VOUCH_RP_ID=auth.example.com \
  -e VOUCH_JWT_SECRET=<your-64-character-secret> \
  -e VOUCH_DATABASE_URL=sqlite:/data/vouch.db?mode=rwc \
  -e VOUCH_TLS_CERT=<base64-encoded-certificate> \
  -e VOUCH_TLS_KEY=<base64-encoded-private-key> \
  ghcr.io/vouch-sh/vouch:latest

Docker Compose

# docker-compose.yml
services:
  vouch-server:
    image: ghcr.io/vouch-sh/vouch:latest
    container_name: vouch-server
    restart: unless-stopped
    ports:
      - "443:443"
      - "80:80"
    volumes:
      - vouch-data:/data
    env_file:
      - vouch.env
    environment:
      VOUCH_DATABASE_URL: sqlite:/data/vouch.db?mode=rwc
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  vouch-data:

Create a vouch.env file:

VOUCH_RP_ID=auth.example.com
VOUCH_JWT_SECRET=<your-64-character-secret>
VOUCH_TLS_CERT=<base64-encoded-certificate>
VOUCH_TLS_KEY=<base64-encoded-private-key>
VOUCH_SSH_CA_KEY=<base64-encoded-ssh-ca-key>

Start:

docker compose up -d
docker compose logs -f vouch-server

With PostgreSQL

# docker-compose.yml
services:
  vouch-server:
    image: ghcr.io/vouch-sh/vouch:latest
    container_name: vouch-server
    restart: unless-stopped
    ports:
      - "443:443"
      - "80:80"
    env_file:
      - vouch.env
    environment:
      VOUCH_DATABASE_URL: postgres://vouch:password@postgres:5432/vouch
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  postgres:
    image: postgres:16
    container_name: vouch-postgres
    restart: unless-stopped
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: vouch
      POSTGRES_USER: vouch
      POSTGRES_PASSWORD: password
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U vouch"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  postgres-data:

Air-Gapped Docker

For air-gapped environments, load the image from a saved archive:

# On connected machine
docker pull ghcr.io/vouch-sh/vouch:1.0.0
docker save ghcr.io/vouch-sh/vouch:1.0.0 -o vouch-server-1.0.0.tar

# Transfer to air-gapped environment

# Load image
docker load < vouch-server-1.0.0.tar

Upgrading

# Pull new image
docker compose pull

# Restart with new image
docker compose up -d

# Verify
docker compose logs -f vouch-server
curl -k https://auth.example.com/health

Kubernetes (Helm)

Deploy Vouch on Kubernetes using the Helm chart.

Prerequisites

• Kubernetes cluster (1.24+)
• Helm 3
• A persistent volume provisioner (for SQLite) or external PostgreSQL

Install

# Install from OCI registry
helm install vouch-server oci://ghcr.io/vouch-sh/charts/vouch-server \
  --version 0.1.0 \
  --namespace vouch \
  --create-namespace \
  --values my-values.yaml

Values

Key values to configure:

# values.yaml
image:
  repository: ghcr.io/vouch-sh/vouch
  pullPolicy: IfNotPresent
  tag: ""  # defaults to chart appVersion

serviceAccount:
  create: true
  annotations: {}

podSecurityContext:
  fsGroup: 65532

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 65532
  seccompProfile:
    type: RuntimeDefault

service:
  type: ClusterIP
  port: 3000

# Environment variables for vouch-server
env:
  VOUCH_LISTEN_ADDR: "0.0.0.0:3000"
  VOUCH_DATABASE_URL: "sqlite:/data/vouch.db?mode=rwc"
  VOUCH_RP_ID: "auth.example.com"
  VOUCH_BASE_URL: "https://auth.example.com"
  RUST_LOG: "info,vouch_server=debug"

# Secret environment variables
# Reference an existing secret containing keys like:
# - VOUCH_JWT_SECRET
# - VOUCH_IDPS                          (e.g., "google")
# - VOUCH_IDP_GOOGLE_TYPE               (oidc|saml)
# - VOUCH_IDP_GOOGLE_ISSUER
# - VOUCH_IDP_GOOGLE_CLIENT_ID
# - VOUCH_IDP_GOOGLE_CLIENT_SECRET
existingSecret: ""

# Or create a new secret (not recommended for production)
secrets: {}
  # VOUCH_JWT_SECRET: ""

# Ingress
ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    - host: auth.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: vouch-tls
      hosts:
        - auth.example.com

# Resources
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 256Mi

# Persistence (for SQLite)
persistence:
  enabled: true
  existingClaim: ""
  storageClass: ""
  accessMode: ReadWriteOnce
  size: 1Gi
  mountPath: /data

# Health check configuration
healthcheck:
  path: /health
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

Using Kubernetes Secrets

Create secrets for sensitive values:

kubectl create secret generic vouch-secrets \
  --namespace vouch \
  --from-literal=VOUCH_JWT_SECRET='<your-64-character-secret>' \
  --from-literal=VOUCH_IDPS='google' \
  --from-literal=VOUCH_IDP_GOOGLE_TYPE='oidc' \
  --from-literal=VOUCH_IDP_GOOGLE_ISSUER='https://accounts.google.com' \
  --from-literal=VOUCH_IDP_GOOGLE_CLIENT_ID='...' \
  --from-literal=VOUCH_IDP_GOOGLE_CLIENT_SECRET='...'

Then reference in values:

existingSecret: vouch-secrets

Air-Gapped Kubernetes

For air-gapped environments:

1. Save and transfer the chart:

   helm pull oci://ghcr.io/vouch-sh/charts/vouch-server --version 0.1.0
   # Transfer vouch-server-0.1.0.tgz to air-gapped environment
   

2. Save and transfer the container image:

   docker pull ghcr.io/vouch-sh/vouch:0.1.0
   docker save ghcr.io/vouch-sh/vouch:0.1.0 -o vouch-0.1.0.tar
   # Transfer and load into your private registry
   

3. Install from the local chart:

   helm install vouch-server ./vouch-server-0.1.0.tgz \
     --namespace vouch \
     --create-namespace \
     --set image.repository=registry.internal/vouch \
     --values my-values.yaml
   

Upgrading

helm upgrade vouch-server oci://ghcr.io/vouch-sh/charts/vouch-server \
  --version <new-version> \
  --namespace vouch \
  --values my-values.yaml

Health Checks

The chart configures liveness and readiness probes against the /health endpoint.

Health Checks and Monitoring

Health Endpoint

Vouch exposes a health check endpoint:

GET /health

Response:

{"status": "healthy"}

This endpoint:

• Returns HTTP 200 when the server is operational
• Is accessible over HTTP (port 80) even when TLS is configured, for load balancer health checks
• Does not require authentication

Monitoring Endpoints

Log Format

Vouch uses structured logging via tracing. Set the log level with the RUST_LOG environment variable:

# Production (warnings and errors only)
RUST_LOG=warn

# Standard operation
RUST_LOG=info

# Debugging
RUST_LOG=debug

# Component-specific logging
RUST_LOG=vouch_server=debug,tower_http=info

Audit Events

Authentication, credential issuance, and administrative events are written to the audit_events table and browsable at /admin/audit. Emails are masked to domain-only, with an HMAC column for correlation.

Authentication and key lifecycle

Credential issuance

OAuth clients

Administration and organization

Retention

Configure retention periods for audit events:

# Auth events (login, enrollment, logout, key/device-auth lifecycle)
# — default 90 days
VOUCH_AUTH_EVENTS_RETENTION_DAYS=730

# OAuth usage and credential-issuance events (oauth_*, aws_credential,
# github_credential, ssh_credential, token_exchange) — default 90 days
VOUCH_OAUTH_EVENTS_RETENTION_DAYS=90

Events older than the retention period are cleaned up automatically by the background cleanup task (controlled by VOUCH_CLEANUP_INTERVAL).

Alerting Recommendations

Identity Provider Overview

Vouch uses one or more upstream identity providers (IdPs) to verify user identity during enrollment. This links a trusted corporate identity to a hardware-bound FIDO2 credential.

Purpose

• Verify the user is a member of your organization during enrollment
• Pull user attributes (email) from your existing identity system
• No separate user database to maintain in Vouch

Supported Protocols

Vouch supports two upstream IdP protocols, configured as a unified list:

Multiple IdPs — of either protocol, in any combination — can be configured simultaneously. The login page renders one “Sign in with X” button per configured IdP, in the order operators listed them.

OIDC Discovery

When using OIDC, the server automatically discovers authorization, token, and JWKS endpoints by fetching the /.well-known/openid-configuration document from the issuer URL at startup. Any OIDC-compliant provider works — no manual endpoint configuration is needed.

Supported Providers

Configuration

IdPs are configured as a unified list. Each IdP has an operator-chosen slug (e.g., google, entra, corp-saml) that becomes its identifier in the state table, login page query string (?provider=<slug>), and audit logs.

Slug rules

• Match [a-z0-9-]{1,32}
• Must not start or end with a hyphen
• Must be unique across all configured IdPs

Environment variables

Set VOUCH_IDPS to a comma-separated list of slugs. For each slug, set VOUCH_IDP_<SLUG>_TYPE to oidc or saml, plus the type-specific variables.

OIDC example (Google + Entra concurrently):

VOUCH_IDPS=google,entra

VOUCH_IDP_GOOGLE_TYPE=oidc
VOUCH_IDP_GOOGLE_ISSUER=https://accounts.google.com
VOUCH_IDP_GOOGLE_CLIENT_ID=<your-google-client-id>
VOUCH_IDP_GOOGLE_CLIENT_SECRET=<your-google-client-secret>

VOUCH_IDP_ENTRA_TYPE=oidc
VOUCH_IDP_ENTRA_ISSUER=https://login.microsoftonline.com/organizations/v2.0
VOUCH_IDP_ENTRA_CLIENT_ID=<your-entra-client-id>
VOUCH_IDP_ENTRA_CLIENT_SECRET=<your-entra-client-secret>

VOUCH_ALLOWED_DOMAINS=company.com

SAML example (mixed alongside OIDC):

VOUCH_IDPS=google,corp-saml

VOUCH_IDP_GOOGLE_TYPE=oidc
VOUCH_IDP_GOOGLE_ISSUER=https://accounts.google.com
VOUCH_IDP_GOOGLE_CLIENT_ID=<your-google-client-id>
VOUCH_IDP_GOOGLE_CLIENT_SECRET=<your-google-client-secret>

VOUCH_IDP_CORP_SAML_TYPE=saml
VOUCH_IDP_CORP_SAML_METADATA_URL=https://idp.example.com/saml/metadata
VOUCH_IDP_CORP_SAML_SP_ENTITY_ID=https://auth.example.com
VOUCH_IDP_CORP_SAML_EMAIL_ATTRIBUTE=http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress
VOUCH_IDP_CORP_SAML_DOMAIN_ATTRIBUTE=department

Notes on slug-to-env-var conversion: hyphens in the slug become underscores in env-var names. corp-saml becomes VOUCH_IDP_CORP_SAML_TYPE, etc.

S3 configuration

In production deployments using S3-backed configuration, IdPs live under the top-level idps array. Each entry has id, type, and type-specific fields:

{
  "idps": [
    {
      "id": "google",
      "type": "oidc",
      "issuer": "https://accounts.google.com",
      "client_id": "<your-google-client-id>",
      "client_secret": "<your-google-client-secret>"
    },
    {
      "id": "corp-saml",
      "type": "saml",
      "metadata_url": "https://idp.example.com/saml/metadata",
      "sp_entity_id": "https://auth.example.com",
      "email_attribute": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
      "domain_attribute": "department"
    }
  ]
}

Order in the idps array controls login-page button order.

Migration from legacy variables

The previous flat single-IdP environment variables (VOUCH_OIDC_ISSUER, VOUCH_OIDC_CLIENT_ID, VOUCH_OIDC_CLIENT_SECRET, VOUCH_OIDC_PROVIDERS, VOUCH_SAML_IDP_METADATA_URL, VOUCH_SAML_SP_ENTITY_ID, VOUCH_SAML_EMAIL_ATTRIBUTE, VOUCH_SAML_DOMAIN_ATTRIBUTE) and the legacy S3 oidc / saml nested blocks are no longer read by the server. They are silently ignored if still present, but they configure nothing — only the unified VOUCH_IDPS / idps[] format below is honored. Update each legacy value to its replacement and remove the old one:

Claims and Attribute Mapping

OIDC Claims

SAML Attributes

User Lifecycle

• User exists in external IdP but not Vouch — Enrollment creates Vouch user
• User removed from external IdP — Existing Vouch sessions continue until expiry; re-enrollment blocked

Google Workspace (OIDC)

Configure Google Workspace as your external identity provider for Vouch enrollment via OpenID Connect.

Prerequisites

• Google Workspace admin access
• A verified domain in Google Workspace

Step 1: Create OAuth Client in Google Cloud Console

1. Go to Google Cloud Console
2. Select or create a project
3. Navigate to APIs & Services > Credentials
4. Click Create Credentials > OAuth client ID
5. Select Web application as the application type
6. Configure:
   • Name: Vouch
   • Authorized redirect URIs: https://auth.example.com/oauth/callback
7. Click Create
8. Copy the Client ID and Client Secret

Step 2: Configure Consent Screen

1. Navigate to APIs & Services > OAuth consent screen
2. Select Internal (restricts to your Google Workspace org)
3. Configure:
   • App name: Vouch
   • User support email: your admin email
   • Authorized domains: your Vouch server domain
4. Add scopes: openid, email, profile

Step 3: Configure Vouch

Add Google to the VOUCH_IDPS list with type oidc:

VOUCH_IDPS=google
VOUCH_IDP_GOOGLE_TYPE=oidc
VOUCH_IDP_GOOGLE_ISSUER=https://accounts.google.com
VOUCH_IDP_GOOGLE_CLIENT_ID=<your-client-id>.apps.googleusercontent.com
VOUCH_IDP_GOOGLE_CLIENT_SECRET=<your-client-secret>

To run Google alongside another IdP (e.g., Microsoft Entra), append both slugs to VOUCH_IDPS — both buttons appear on the login page in list order.

The server automatically discovers Google’s authorization, token, and JWKS endpoints via OIDC Discovery. No manual endpoint configuration is needed.

Optionally restrict enrollment to specific domains:

VOUCH_ALLOWED_DOMAINS=example.com,subsidiary.com

S3 configuration

{
  "idps": [
    {
      "id": "google",
      "type": "oidc",
      "issuer": "https://accounts.google.com",
      "client_id": "<your-client-id>.apps.googleusercontent.com",
      "client_secret": "<your-client-secret>"
    }
  ]
}

Step 4: Test

1. Run vouch enroll on a workstation
2. The browser should redirect to Google sign-in
3. After signing in, complete the WebAuthn registration with your YubiKey

Claims Mapping

Troubleshooting

“Access blocked: This app’s request is invalid”

• Verify the redirect URI exactly matches https://<your-vouch-domain>/oauth/callback

“This app is not verified”

• Ensure the OAuth consent screen is set to Internal for Google Workspace

Users from wrong domain can enroll

• Set VOUCH_ALLOWED_DOMAINS to restrict enrollment to specific email domains

Microsoft Entra ID (OIDC)

Configure Microsoft Entra ID (formerly Azure AD) as your upstream identity provider for Vouch enrollment.

Prerequisites

• Microsoft Entra ID tenant with admin access
• An app registration in the Azure portal

Step 1: Register an Application in Entra ID

Follow Microsoft’s app registration guide to create a new app registration:

1. Sign in to the Azure portal
2. Navigate to Microsoft Entra ID > App registrations > New registration
3. Configure:

   • Name: Vouch

   • Supported account types: choose one of:

     • Accounts in this organizational directory only (single-tenant) — restrict to your tenant.
     • Accounts in any organizational directory (Any Microsoft Entra ID tenant – Multitenant) — let users from any work/school tenant enroll.

     Do not select “Accounts in any organizational directory and personal Microsoft accounts” or “Personal Microsoft accounts only”. Personal Microsoft accounts (outlook.com, hotmail.com, live.com) are not supported by Vouch — see Why personal accounts aren’t supported below.

   • Redirect URI: Select Web and enter https://auth.example.com/oauth/callback

4. Click Register

Step 2: Create a Client Secret

1. In the app registration, go to Certificates & secrets > Client secrets
2. Click New client secret
3. Set a description and expiry period
4. Copy the Value (not the Secret ID) immediately — it is only shown once

Step 3: Configure the xms_edov optional claim

Vouch requires the xms_edov (Email Domain Owner Verified) claim to confirm the user’s email is admin-verified by their tenant. Microsoft does not emit this claim by default — you must add it as an optional claim:

1. In the app registration, go to Token configuration > Add optional claim
2. Select token type ID
3. Check xms_edov
4. Click Add
5. If prompted to “Turn on the Microsoft Graph email permission”, accept

The next ID token Microsoft issues will contain "xms_edov": true when the email’s domain is verified by the user’s tenant admin.

Step 4: Configure Vouch

Add Entra to the VOUCH_IDPS list with type oidc:

Single-tenant

VOUCH_IDPS=entra
VOUCH_IDP_ENTRA_TYPE=oidc
VOUCH_IDP_ENTRA_ISSUER=https://login.microsoftonline.com/{tenant-id}/v2.0
VOUCH_IDP_ENTRA_CLIENT_ID=<application-client-id>
VOUCH_IDP_ENTRA_CLIENT_SECRET=<client-secret-value>

Replace {tenant-id} with your Entra ID tenant ID (found in Azure portal > Microsoft Entra ID > Overview).

Multi-tenant (any work/school tenant)

VOUCH_IDP_ENTRA_ISSUER=https://login.microsoftonline.com/organizations/v2.0

Vouch handles the {tenantid} template issuer that /organizations/ discovery returns, then cross-checks the per-tenant tid claim in each ID token against the tenant UUID in the token’s iss claim to prevent cross-tenant token injection.

The server automatically discovers authorization, token, and JWKS endpoints from the issuer URL via OIDC Discovery. No manual endpoint configuration is needed.

Optionally restrict enrollment to specific email domains:

VOUCH_ALLOWED_DOMAINS=example.com

S3 configuration

{
  "idps": [
    {
      "id": "entra",
      "type": "oidc",
      "issuer": "https://login.microsoftonline.com/{tenant-id}/v2.0",
      "client_id": "<application-client-id>",
      "client_secret": "<client-secret-value>"
    }
  ]
}

Step 5: Test

1. Run vouch enroll on a workstation
2. The browser should redirect to the Microsoft sign-in page
3. After signing in, complete the WebAuthn registration with your YubiKey

Why personal Microsoft accounts aren’t supported

Vouch issues hardware-backed credentials and tracks users by verified email address. To verify an Entra email, Vouch requires the xms_edov optional claim, which provides a signed assertion that the email’s domain is admin-verified.

Microsoft only emits optional claims (including xms_edov) for app registrations that target Microsoft Entra ID accounts only. Per Microsoft’s app manifest reference:

Apps that support both personal accounts and Microsoft Entra ID can’t use optional claims.

As a consequence, the https://login.microsoftonline.com/common/v2.0 issuer is rejected at startup — Vouch will refuse to load an Entra IdP configured with /common/. Use /organizations/v2.0 or a single-tenant URL instead.

Common Pitfalls

InvalidIssuer at the callback

Make sure the configured issuer ends in /v2.0. The v1 endpoints use a different token format and are not compatible with standard OIDC discovery.

Email address is not verified by the identity provider after sign-in

The xms_edov optional claim isn’t configured on the app registration. Repeat Step 3. If the claim is already configured but the user’s email still isn’t accepted, the user’s tenant admin has not verified the email domain — Microsoft only sets xms_edov: true when the domain is verified inside the user’s tenant.

Redirect URI mismatch

The redirect URI in the app registration must exactly match https://<your-vouch-domain>/oauth/callback. Azure does not support wildcard redirect URIs.

Multi-tenant without an allowlist

If you use /organizations/v2.0, any Entra tenant can attempt enrollment. Set VOUCH_ALLOWED_DOMAINS to restrict which email domains Vouch will accept.

Generic OIDC Provider

Configure any OpenID Connect-compliant identity provider as the upstream IdP for Vouch enrollment.

Prerequisites

• Your IdP must support OIDC Discovery (a /.well-known/openid-configuration endpoint at the issuer URL)
• You need a registered OAuth 2.0 client (client ID and client secret)
• The redirect URI https://<your-vouch-domain>/oauth/callback must be registered with the IdP

Finding the Issuer URL

The issuer URL is the base URL that hosts the OIDC discovery document. You can verify it by fetching {issuer}/.well-known/openid-configuration and confirming it returns a valid JSON document:

curl -s https://your-idp.example.com/.well-known/openid-configuration | jq .issuer

Common issuer URL patterns:

Configuration

Pick a slug for the IdP (e.g., okta, keycloak, auth0-corp) and add it to VOUCH_IDPS. Slug rules: [a-z0-9-]{1,32}, no leading or trailing hyphen, unique across IdPs.

VOUCH_IDPS=okta
VOUCH_IDP_OKTA_TYPE=oidc
VOUCH_IDP_OKTA_ISSUER=https://your-idp.example.com
VOUCH_IDP_OKTA_CLIENT_ID=<your-client-id>
VOUCH_IDP_OKTA_CLIENT_SECRET=<your-client-secret>

Hyphens in the slug become underscores in env-var names: a slug of auth0-corp becomes VOUCH_IDP_AUTH0_CORP_*.

At startup, the server fetches the discovery document from {issuer}/.well-known/openid-configuration and automatically discovers the authorization, token, and JWKS endpoints. No manual endpoint configuration is needed.

S3 configuration

{
  "idps": [
    {
      "id": "okta",
      "type": "oidc",
      "issuer": "https://your-idp.example.com",
      "client_id": "<your-client-id>",
      "client_secret": "<your-client-secret>"
    }
  ]
}

Domain Restrictions

Restrict enrollment to specific email domains:

VOUCH_ALLOWED_DOMAINS=example.com,subsidiary.com

If not set, users from any email domain can enroll (provided they authenticate with the upstream IdP).

Tested Providers

The following providers have been tested with Vouch:

Troubleshooting

“Failed to fetch upstream OIDC discovery document”

• Verify the issuer URL is correct and reachable from the server
• Check that the URL uses HTTPS (HTTP is only allowed for localhost)
• Confirm the discovery endpoint returns valid JSON

“Issuer mismatch”

• The issuer field in the discovery document must exactly match the configured VOUCH_IDP_<SLUG>_ISSUER value (trailing slashes matter). Entra /organizations/v2.0 is special-cased automatically — its discovery document returns a {tenantid} template that vouch handles transparently. The /common/v2.0 endpoint is not supported; see Microsoft Entra ID.

Token errors after authentication

• Ensure the client secret is correct and not expired
• Verify the redirect URI registered with the IdP exactly matches https://<your-vouch-domain>/oauth/callback

SAML 2.0

Configure one or more SAML 2.0 identity providers for Vouch enrollment. Vouch acts as a SAML Service Provider (SP) with HTTP-POST and HTTP-Redirect bindings.

SAML IdPs are configured under the same VOUCH_IDPS list as OIDC IdPs and can coexist with OIDC providers in any combination.

Environment Variables

Pick a slug for the IdP (e.g., corp-saml, partner-saml) and add it to VOUCH_IDPS. Slug rules: [a-z0-9-]{1,32}, no leading/trailing hyphen, unique across IdPs.

Hyphens in the slug become underscores in env-var names: a slug of corp-saml becomes VOUCH_IDP_CORP_SAML_*.

SP Metadata

The Vouch server exposes SP metadata for configuring your IdP:

GET https://auth.example.com/saml/metadata

This returns an XML document containing the SP entity ID, Assertion Consumer Service (ACS) URL, and supported bindings. The SP entity ID comes from the first configured SAML IdP (or the server’s base URL if none is set). Import the metadata into your IdP or configure the following values manually:

• SP Entity ID: https://auth.example.com (or the value of VOUCH_IDP_<SLUG>_SP_ENTITY_ID)
• ACS URL: https://auth.example.com/saml/acs
• Bindings: HTTP-POST (ACS), HTTP-Redirect (AuthnRequest)

All configured SAML IdPs share the single ACS URL; Vouch identifies the originating IdP via the per-request RelayState stored in the state table.

Attribute Mapping

Vouch extracts user identity from SAML assertion attributes. By default, it looks for the email address in common attribute names. You can override this per-IdP with VOUCH_IDP_<SLUG>_EMAIL_ATTRIBUTE:

For domain-based enrollment restrictions, set VOUCH_IDP_<SLUG>_DOMAIN_ATTRIBUTE to the attribute name containing the user’s domain. If not set, the domain is extracted from the email address.

Configuration Example

VOUCH_IDPS=corp-saml

VOUCH_IDP_CORP_SAML_TYPE=saml
VOUCH_IDP_CORP_SAML_METADATA_URL=https://login.microsoftonline.com/{tenant-id}/federationmetadata/2007-06/federationmetadata.xml
VOUCH_IDP_CORP_SAML_SP_ENTITY_ID=https://auth.example.com
VOUCH_IDP_CORP_SAML_EMAIL_ATTRIBUTE=http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress

VOUCH_ALLOWED_DOMAINS=example.com

S3 configuration

{
  "idps": [
    {
      "id": "corp-saml",
      "type": "saml",
      "metadata_url": "https://login.microsoftonline.com/{tenant-id}/federationmetadata/2007-06/federationmetadata.xml",
      "sp_entity_id": "https://auth.example.com",
      "email_attribute": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"
    }
  ]
}

Provider-Specific Notes

Microsoft Entra ID

• Metadata URL: https://login.microsoftonline.com/{tenant-id}/federationmetadata/2007-06/federationmetadata.xml
• Email attribute: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress
• Create an Enterprise Application with SAML SSO and set the ACS URL to https://auth.example.com/saml/acs

Okta

• Metadata URL: found in the Okta SAML app’s Sign On tab under Metadata URL
• Email attribute: typically email or configure in the Okta attribute statements
• Set the Single Sign-On URL to https://auth.example.com/saml/acs and Audience URI to your SP entity ID

Google Workspace

• Metadata URL: available from the Google Admin console under Apps > Web and mobile apps > SAML app > Metadata
• Email attribute: email (configure in attribute mapping)
• Add a custom SAML app in the Admin console with ACS URL https://auth.example.com/saml/acs

Troubleshooting

“Failed to fetch SAML IdP metadata”

• Verify the metadata URL is correct and reachable from the server
• Check that the URL returns valid XML (not an HTML login page)
• Ensure the server can make outbound HTTPS requests

Signature verification errors

• Confirm the IdP’s signing certificate in the metadata is current and not expired
• Ensure the server clock is synchronized (NTP). SAML assertions have time-based validity windows.

Email not extracted from assertion

• Check the SAML assertion attributes using debug logging (RUST_LOG=vouch_server=debug)
• Set VOUCH_IDP_<SLUG>_EMAIL_ATTRIBUTE to the exact attribute name used by your IdP

“Legacy identity-provider configuration detected”

• The flat VOUCH_SAML_* variables are no longer accepted. Migrate to the per-IdP VOUCH_IDP_<SLUG>_* format. See Overview for the full mapping.

Session Management

Vouch sessions are time-limited, DPoP-bound OAuth 2.0 access tokens (ES256 JWTs per RFC 9068) that prove recent hardware presence verification.

Session Lifecycle

1. Creation — vouch login performs FIDO2 assertion with YubiKey touch + PIN
2. Active — Access token stored in agent memory, valid for 8 hours (default)
3. Usage — Credential helpers exchange the access token for service-specific credentials
4. Expiry — Session expires automatically after the configured duration
5. Revocation — vouch logout explicitly ends the session

Session Duration

Default: 8 hours. Configurable via:

VOUCH_SESSION_HOURS=8

Session Storage

Sessions are stored in multiple locations for different access patterns:

Vouch follows the XDG Base Directory Specification and honors XDG_CONFIG_HOME, XDG_STATE_HOME, XDG_DATA_HOME, XDG_CACHE_HOME, and XDG_RUNTIME_DIR on all platforms (including macOS). The paths above show the defaults when those variables are unset. See File Locations for the full map and the automatic migration from the legacy ~/.vouch/ directory.

Server-Side Session Management

Cleanup

Expired sessions are cleaned up automatically by a background task:

# Cleanup interval in minutes (default: 15, set to 0 to disable)
VOUCH_CLEANUP_INTERVAL=15

Security Properties

• Presence-bound — Every session traces to a FIDO2 assertion with user verification
• Time-limited — Sessions cannot be renewed; a new login is required after expiry
• DPoP-bound — Access tokens are bound to the client’s DPoP key; token theft without the key is useless
• Non-transferable — Sessions are bound to the client that created them
• Audience-restricted — Tokens narrowed to a specific resource are rejected at every other resource (see below)
• Audited — Every session creation and usage is logged

Audience Enforcement (RFC 8707 Resource Indicators)

Access tokens carry an aud (audience) claim per RFC 9068. By default the audience equals the requesting client_id and the token is valid at every Vouch resource endpoint — this covers all standard flows (vouch login, browser sessions, device flow, client credentials).

A client may instead narrow a token to a specific resource, either with the RFC 8707 resource parameter at the authorization endpoint or with the audience/resource parameters at token exchange (RFC 8693). Vouch’s resource endpoints (/v1/credentials/*, /v1/keys, /api/v1/*, RFC 7592 client management) enforce that narrowing: a narrowed token is accepted only when its audience names this deployment (same scheme, host, and port as the configured base URL) and its path covers the request at a path-segment boundary. An audience of the deployment root (the base URL itself) covers every endpoint; {base_url}/v1/keys covers /v1/keys and everything below it, but nothing else. Requests failing the check receive 401 invalid_token with the standard WWW-Authenticate challenge, and the rejection is logged with the client ID, audience, and request path.

Per their RFCs, the authorization-server endpoints remain audience-agnostic: /oauth/userinfo accepts tokens from any client, /oauth/introspect and /oauth/revoke answer about any token the server issued, and token exchange accepts narrowed subject tokens (re-scoping them is its purpose).

Clients registered without resource_uris may request any resource value at issuance. This is safe under enforcement: a token narrowed to an external resource server is less usable at Vouch, not more — it can only be spent at the external service it names. Registering resource_uris additionally restricts which values a client may request at all.

Key Management

Vouch uses several cryptographic keys. This page covers their lifecycle and rotation.

Key Inventory

All of these keys use classical (pre-quantum) algorithms. For why that is currently the right choice for each of them, and what Vouch already does about quantum resistance, see Post-Quantum Cryptography.

SSH CA Key

The SSH CA key signs all SSH user certificates. Every host that trusts Vouch certificates must have the corresponding public key in TrustedUserCAKeys.

Generation

ssh-keygen -t ed25519 -f ssh_ca_key -N "" -C "vouch-ca@example.com"

Configuration

# Option 1: File path
VOUCH_SSH_CA_KEY_PATH=./ssh_ca_key

# Option 2: Inline (base64-encoded PEM, takes precedence over file)
VOUCH_SSH_CA_KEY="$(base64 -i ssh_ca_key | tr -d '\n')"

# Option 3: AWS KMS (overrides Options 1 and 2)
VOUCH_SSH_CA_KMS_KEY_ID=mrk-1234abcd5678efgh

# Option 4: Disable SSH CA
VOUCH_SSH_CA_KEY_PATH=""

When using KMS, the server calls kms:Sign with Ed25519. The KMS key must be an asymmetric signing key with ECC_EDWARDS_CURVE_25519 key spec. Multi-region keys (mrk- prefix) are recommended for high availability.

Rotation

SSH CA key rotation requires coordinated updates:

1. Generate a new CA key
2. Distribute the new public key to all hosts (add to TrustedUserCAKeys)
3. Update the Vouch server configuration with the new private key
4. Restart the server
5. After all existing certificates expire (max 8 hours), remove the old public key from hosts

Important: During rotation, hosts should trust both old and new CA public keys to avoid disruption.

Public Key Distribution

Retrieve the CA public key:

curl https://auth.example.com/v1/credentials/ssh/ca
# ssh-ed25519 AAAA... vouch-ca@example.com

OIDC Signing Key (ES256)

Used to sign access tokens (RFC 9068) and ID tokens (default algorithm) with ES256.

Configuration

# Option 1: Local key (base64-encoded PEM)
VOUCH_OIDC_SIGNING_KEY="$(base64 -i oidc_signing_key.pem | tr -d '\n')"

# Option 2: AWS KMS (overrides Option 1)
VOUCH_OIDC_SIGNING_KMS_KEY_ID=mrk-abcd1234efgh5678

If neither is set, an ephemeral key is generated on startup. This means tokens cannot be verified after a server restart unless the same key is provided.

When using KMS, the server calls kms:Sign with P-256 ECDSA (ECC_NIST_P256 key spec). Multi-region keys (mrk- prefix) are recommended.

Generation

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:prime256v1 -out oidc_signing_key.pem

Note: You must use openssl genpkey (which produces PKCS#8 format) rather than openssl ecparam -genkey (which produces SEC1 format). The server requires PKCS#8 (-----BEGIN PRIVATE KEY-----).

Rotation

When rotating the OIDC signing key:

1. Generate a new key
2. Update the server configuration
3. Restart the server
4. The JWKS endpoint (/oauth/jwks) automatically serves the new public key
5. Relying parties that cache JWKS will pick up the new key on their next refresh

OIDC RSA Signing Key (RS256)

Used to sign ID tokens with RS256 algorithm per OIDC Core Section 3.1.3.7 and all AWS credential tokens. RS256 is the default id_token_signed_response_alg in the OIDC specification and must be supported for conformance. Clients can select RS256 via OAuth 2.0 Dynamic Client Registration (id_token_signed_response_alg field). The AWS token endpoint (/v1/credentials/aws/token) issues one RS256-signed token that serves both STS AssumeRoleWithWebIdentity and, as the sso-oidc:CreateTokenWithIAM assertion, the IAM Identity Center trusted-token-issuer contract (which rejects ES256).

Access tokens are always signed with ES256 (the OIDC Signing Key above).

Generation

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out oidc_rsa_key.pem

A minimum key size of 3072 bits is enforced. Keys smaller than 3072 bits are rejected at startup.

Configuration

# Option 1: Local key (base64-encoded PEM)
VOUCH_OIDC_RSA_SIGNING_KEY="$(base64 -i oidc_rsa_key.pem | tr -d '\n')"

# Option 2: AWS KMS (overrides Option 1)
VOUCH_OIDC_RSA_SIGNING_KMS_KEY_ID=mrk-rsa1234abcd5678

If neither is set, an ephemeral RSA-3072 key is generated on startup. This means RS256 ID tokens and AWS credential tokens cannot be verified after a server restart, and verification fails across multiple instances (each generates its own key). Any deployment using the AWS integration needs a durable key. A warning is logged when an ephemeral key is generated.

When using KMS, the key must be:

• Key spec: RSA_3072
• Key usage: SIGN_VERIFY
• Signing algorithm: RSASSA_PKCS1_V1_5_SHA_256

Multi-region keys (mrk- prefix) are recommended.

Rotation

When rotating the OIDC RSA signing key:

1. Generate a new RSA-3072 key
2. Update the server configuration
3. Restart the server
4. The JWKS endpoint (/oauth/jwks) automatically serves the new public key
5. Relying parties that cache JWKS will pick up the new key on their next refresh

JWT Secret

Used for signing internal state tokens (authorization codes, WebAuthn challenge state, CSRF tokens) with HS256. Access tokens are signed with the OIDC signing key (ES256) per RFC 9068.

Configuration

# Option 1: Local secret (must be at least 32 characters)
VOUCH_JWT_SECRET="$(openssl rand -base64 48)"

# Option 2: AWS KMS HMAC (eliminates the need for VOUCH_JWT_SECRET)
VOUCH_JWT_HMAC_KMS_KEY_ID=mrk-5678abcd1234efgh

When using KMS, the server uses kms:GenerateMac and kms:VerifyMac with HMAC-SHA256. The KMS key must be a HMAC_256 key type. Multi-region keys (mrk- prefix) are recommended.

Generation (local secret)

openssl rand -base64 48

Rotation

Changing the JWT secret (or KMS key) invalidates all existing sessions. Users must re-authenticate.

1. Generate a new secret or KMS key
2. Update VOUCH_JWT_SECRET or VOUCH_JWT_HMAC_KMS_KEY_ID
3. Restart the server
4. All users must run vouch login again

Document Encryption Key

Used for HPKE (Hybrid Public Key Encryption) of sensitive documents stored alongside the S3 configuration. The private key is encrypted by a KMS key and stored in the S3 config as document_key.

Provisioning

vouch-server generate-document-key --kms-key-id mrk-<your-kms-key-id>

This generates a P-384 EC key pair, encrypts the private key with the specified KMS key, and outputs the document_key JSON block to add to your S3 config. --algorithm p384 is the default and currently the only supported algorithm; the flag exists so post-quantum algorithms can be added later without changing the command or config shape.

Configuration

The document_key field in S3 config contains:

{
  "document_key": {
    "kms_key_id": "mrk-<your-kms-key-id>",
    "encrypted_private_key": "<base64-encoded KMS ciphertext>",
    "algorithm": "p384"
  }
}

algorithm is optional and defaults to p384, so configs provisioned before the field existed keep working unchanged.

At startup, the server decrypts the private key via kms:Decrypt and holds the key material in memory for the lifetime of the process.

Cipher-suite tagging and the post-quantum path

Every document row records the HPKE cipher suite it was sealed with: the stored encapsulated key is prefixed hpke:<kem_id>:<kdf_id>:<aead_id>: using the RFC 9180 codepoints (hpke:0011:0002:0002: for the current DHKEM(P-384) + HKDF-SHA384 + AES-256-GCM suite). Rows written before tagging existed are plain base64 and are read as that same P-384 suite. Rows sealed under different suites can therefore coexist in one database, which is what makes a future key-encapsulation migration — e.g. to the ML-KEM hybrid suites from draft-ietf-hpke-pq — an operational rotation rather than a breaking format change.

There is no document-key rotation mechanism today. When post-quantum suites become available in the underlying libraries (rustls / aws-lc-rs), the expected migration is:

1. Provision a new document_key with the new algorithm (one new generate-document-key --algorithm value).
2. Run a dual-key read period: the server decrypts old rows with the old private key (selected by each row’s suite tag) while sealing new writes under the new suite.
3. Re-encrypt existing rows opportunistically on write, plus an offline sweep for the remainder; then retire the old key.

Steps 2–3 are not implemented yet — only the storage format and configuration groundwork exist. Do not remove the old key from KMS until every row carries the new suite tag.

Per-Org Issuer Signing Keys (ES256 + RS256)

When an organization claims a custom subdomain on an encrypted deployment, Vouch generates a dedicated ES256 and RS256 signing key pair for that org’s OIDC issuer. AWS federation tokens, Identity Center tokens, and all RFC 8693 token-exchange assertions for that org are signed with these keys and served at the org’s own JWKS endpoint (https://<org-subdomain>.auth.example.com/oauth/jwks).

This makes each subdomain a real cryptographic tenant boundary: a token issued for org A cannot be verified against org B’s JWKS.

Per-org keys are only created when all three conditions are met:

• The deployment has document encryption enabled (a KMS-backed document key in the S3 config).
• The organization has a claimed subdomain.
• A credential-issuance request arrives for that org (lazy first-use creation).

Key Lifecycle

Each algorithm’s key set always contains two keys, and sometimes three:

The Next key is created together with the first key and re-staged automatically whenever a rotation consumes it, so relying-party JWKS caches always hold the key that will sign next — long before it ever signs. Nothing in the lifecycle runs on a timer: both rotation steps are explicit operator actions on the Admin → Subdomain page.

Rotating Keys

Step 1 — Rotate. The “Rotate Signing Keys” button switches signing to the pre-staged Next keys for both algorithms in one transaction. The old signers become Previous keys: still published, still verifying outstanding tokens, no longer signing. Fresh Next keys are staged in the same transaction.

The rotate is rejected in two situations:

• The Next keys are younger than 24 hours. Relying parties (AWS IAM in particular) cache the org JWKS on their own schedule; signing with a key their cache has not seen fails federation until they refetch. Because the Next key is normally staged months earlier (at first use or by the previous rotation), this gate only bites on back-to-back rotations.
• Previous keys from an earlier rotation are still published. Revoke them first — the key set keeps at most one retired generation per algorithm.

Step 2 — Revoke. The “Revoke Old Keys” button deletes the Previous keys and removes them from the JWKS. It is rejected until max(session lifetime, 8 hours) + 2 hours have passed since the rotate, because until then tokens signed by the old keys may still be live — deleting the keys would log those sessions out. After the window, revocation affects nobody.

A Previous key that is never revoked stays visible on the admin page indefinitely (it can verify, but never sign). Nothing deletes it automatically; revoking promptly after the drain window keeps the published key set minimal.

Operator note: Reducing VOUCH_SESSION_HOURS between a rotate and its revoke can shorten the revoke gate below what tokens issued under the old lifetime need. Revoke first, then shorten session lifetimes.

Emergency Rotation

The “Emergency Rotate” button replaces the entire key set — fresh Current and Next keys for both algorithms, Previous keys deleted — in one atomic operation. Use it only when key compromise is suspected: on an encrypted deployment all private keys are sealed by the same document key, so a compromise of one is treated as a compromise of all.

Consequences:

• Every key that existed before the emergency is removed from the JWKS immediately.
• Outstanding tokens signed by the old keys will fail verification until relying parties refetch the JWKS. Cross-instance propagation takes up to 60 seconds (signing cache TTL); downstream relying parties that respect the Cache-Control: public, max-age=3600 response header may take up to 1 hour to pick up the new keys.
• AWS STS AssumeRoleWithWebIdentity and IAM Identity Center CreateTokenWithIAM calls that carry a token signed by an old key will fail until the user re-authenticates with vouch login.
• The fresh Next keys start a new 24-hour publish window, so a graceful rotate is unavailable for a day afterwards.

Runbook:

1. Navigate to Admin → Subdomain for the affected org.
2. Click Emergency Rotate and confirm.
3. Instruct affected users to run vouch login to obtain a new token signed by the replacement key.
4. If the org is federated with AWS IAM, existing STS sessions will expire naturally (up to session lifetime) or can be revoked via the IAM console.

JWKS Caching and the 24-Hour Publish Window

The 24-hour minimum age before a Next key may sign is a deliberate product decision. AWS IAM and IAM Identity Center cache JWKS responses for an undocumented internal period that is believed to exceed the advertised 1-hour Cache-Control max-age. Keeping the successor published for at least 24 hours before it signs ensures relying parties have ample time to cache the new kid. Changing this window requires verifying the behaviour of all federated relying parties.

Releasing a subdomain deletes the Next and Previous keys (the publish window is meaningless while the issuer host is unclaimed) but keeps the Current key, so a same-org reclaim resumes with the same signer. The first use after a reclaim stages a fresh Next key, which restarts its 24-hour window.

TLS Certificate

See TLS Configuration for details on TLS certificate management and hot-reload.

Post-Quantum Cryptography

This page describes Vouch’s post-quantum cryptography (PQC) posture: what is quantum-resistant today, what is still classical and why, and what to watch. It exists so that operators fielding “are we quantum-safe?” questions have a precise answer.

Threat Model Summary

A cryptographically relevant quantum computer breaks the elliptic-curve and RSA algorithms Vouch uses (P-256, P-384, Ed25519, RSA-3072). The two exposure classes have very different urgency:

• Key exchange / encryption — urgent. An adversary can record encrypted traffic or steal encrypted data today and decrypt it once a quantum computer exists (“harvest now, decrypt later”). This is why the industry moved TLS and SSH key exchange to hybrid post-quantum schemes first (NIST IR 8547; OpenSSH 10 defaults; the majority of browser traffic).
• Signatures — not yet exposed. A signature is verified at use time. A future quantum computer cannot retroactively forge a login, token, or certificate that was verified years earlier. Signatures only become a problem once quantum computers are close to real, and the surrounding ecosystems (JOSE, OpenSSH, FIDO2, WebPKI) have not yet standardized replacements.

What Is Quantum-Resistant Today

TLS key exchange, in both directions. All Vouch binaries enable the rustls prefer-post-quantum feature, which makes X25519MLKEM768 (hybrid X25519 + ML-KEM-768, the FIPS 203 lattice KEM) the preferred TLS key-exchange group:

• Inbound: the server’s HTTPS and mTLS listeners.
• Outbound: CLI → server, agent → server, and server → upstream IdP / AWS / GitHub connections.

Hybrid means the session keys are secure if either X25519 or ML-KEM-768 holds, so this is a strict upgrade. Peers that do not support ML-KEM negotiate classical groups transparently. No configuration is involved; an integration test (pq_tls.rs in vouch-tests) pins the negotiated group so a silent downgrade fails CI.

What Is Still Classical, and Why

Every signature in the system is classical, because none of the protocols involved have a standardized, deployable post-quantum alternative yet. None of these are “harvest now, decrypt later” exposed.

One encryption surface is classical and is the known harvest-now-decrypt-later item:

Watch List

Ecosystem developments that unblock the classical surfaces above, roughly in the order they are expected to land:

1. HPKE hybrid KEMs (X-Wing / ML-KEM in HPKE) — unblocks document encryption at rest. First candidate for adoption since it is entirely under Vouch’s control (no protocol peers involved).
2. ML-DSA in JOSE/COSE (draft-ietf-cose-dilithium → RFC, plus Rust library support) — unblocks OIDC tokens, DPoP, and client assertions. AWS KMS already offers ML-DSA signing keys, so the KMS-backed key path is the likely first implementation.
3. OpenSSH post-quantum signature algorithms — unblocks the SSH CA. Nothing to do until OpenSSH itself ships them.
4. FIDO2/CTAP PQC + authenticator hardware — unblocks WebAuthn. Requires new YubiKey hardware generations; expect years, and expect hybrid classical+PQC attestation first.
5. ML-DSA in WebPKI certificates — affects TLS server certificates, which Vouch consumes rather than issues; adopt whatever the CA ecosystem (e.g. Let’s Encrypt) ships.

When adopting any of these, prefer hybrid (classical + PQC) constructions over pure PQC, matching current NIST/BSI guidance.

Operator FAQ

Do I need to configure anything to get post-quantum TLS? No. It is compiled in and negotiated automatically when the peer supports it.

How do I verify a connection used post-quantum key exchange? From a client with OpenSSL 3.5+: openssl s_client -connect auth.example.com:443 -groups X25519MLKEM768 and check the negotiated group in the handshake output. Browsers show the key-exchange group in their developer-tools security panel.

Are my SSH certificates quantum-safe? The certificate signature is Ed25519 (classical), but certificates live at most 8 hours and cannot be retroactively forged, so there is no stored-data exposure. The SSH transport is between your SSH client and your hosts — OpenSSH 10 uses post-quantum key exchange by default.

Does the 8-hour credential lifetime matter here? Yes — short-lived credentials are themselves a quantum mitigation. Even aggressive estimates for cryptographically relevant quantum computers are measured in years, not hours; nothing Vouch signs outlives the transition window it would need to survive to be at risk.

SCIM Provisioning

Vouch supports SCIM 2.0 (RFC 7643/7644) for user provisioning and de-provisioning from external identity providers. SCIM is a launch requirement for enterprise deployments.

Setup

The admin API endpoints (/api/v1/org/*) require an authenticated Vouch session from a user with org admin privileges. The server accepts the access token via Authorization: Bearer <token>, Authorization: DPoP <token>, or the vouch_session cookie.

Prerequisites:

• Your user account must have the is_org_admin flag set in the database
• You must be assigned to an organization (org_id)

1. Create a SCIM token

Log in with vouch login, then create a SCIM token using the cookie file or token command:

# Using cookie file (written automatically on login)
curl -X POST https://auth.example.com/api/v1/org/scim-tokens \
  -b ~/.local/state/vouch/cookie.txt \
  -H "Content-Type: application/json" \
  -d '{"description": "SCIM integration", "expires_in_days": 90}'

# Or using the token command
curl -X POST https://auth.example.com/api/v1/org/scim-tokens \
  -H "Authorization: Bearer $(vouch credential token)" \
  -H "Content-Type: application/json" \
  -d '{"description": "SCIM integration", "expires_in_days": 90}'

The response includes a token prefixed vouch_scim_.... This token is shown once and cannot be retrieved again.

2. Configure your IdP

Enter the following in your IdP’s SCIM configuration:

• SCIM endpoint URL: https://auth.example.com/scim/v2/
• Bearer token: the vouch_scim_... token from step 1

3. Manage tokens

# List active SCIM tokens
curl -b ~/.local/state/vouch/cookie.txt \
  https://auth.example.com/api/v1/org/scim-tokens

# Revoke a SCIM token
curl -X DELETE -b ~/.local/state/vouch/cookie.txt \
  https://auth.example.com/api/v1/org/scim-tokens/<token-id>

De-Provisioning Behavior

When a user is de-provisioned via SCIM (e.g., employee leaves the organization):

Key principle: De-provisioning is immediate and complete. When someone leaves via SCIM, they lose all Vouch access instantly — no waiting for session expiration.

#![allow(unused)]
fn main() {
// SCIM de-provision handling (DELETE /scim/v2/Users/:id)
async fn delete_user(user_id: &str) -> Result<()> {
    // 1. Invalidate all active sessions immediately
    db::delete_sessions_for_user(&db, user_id).await?;

    // 2. Revoke all SSH certificates for this user
    db::revoke_all_ssh_certificates_for_user(&db, user_id, Some("User deleted via SCIM"), Some("scim")).await?;

    // 3. Delete user (cascades to authenticators)
    db::delete_user(&db, user_id).await?;

    // 4. Log audit event
    db::insert_scim_audit(&db, "delete", "User", user_id, Some(&token_id), Some(&details)).await?;

    Ok(())
}
}

SCIM Endpoint Authentication

SCIM endpoints require bearer token authentication:

Endpoint: POST /scim/v2/Users, DELETE /scim/v2/Users/:id, etc.

Authentication:

• Bearer token in Authorization header
• Token generated via the admin API (POST /api/v1/org/scim-tokens)
• Tokens are long-lived but can be rotated/revoked
• Separate token per IdP integration

# Example SCIM request
curl -X DELETE https://vouch.example.com/scim/v2/Users/usr_abc123 \
  -H "Authorization: Bearer scim_token_xyz789" \
  -H "Content-Type: application/scim+json"

Token Security:

• Tokens are hashed (SHA-256) before storage
• Shown once at creation, never retrievable after
• Bound to specific organization
• Minimum 256 bits of entropy

SCIM Audit Logging

All SCIM operations are logged for compliance and security monitoring:

SCIM vs Manual Enrollment

Note: SCIM pre-provisioning creates a user record, but they still cannot authenticate until they physically enroll a hardware FIDO2 authenticator. The security model remains: no credential without hardware.

S3 Configuration Storage

When using S3-based configuration, additional security considerations apply. This chapter covers bucket requirements, protected fields, polling behavior, and key encoding.

S3 Bucket Requirements

Recommended S3 Bucket Policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::ACCOUNT:role/vouch-server"},
      "Action": ["s3:GetObject", "s3:HeadObject"],
      "Resource": "arn:aws:s3:::my-bucket/config/vouch-server.json"
    }
  ]
}

Protected Configuration Fields

Only TLS certificates can be hot-reloaded at runtime via S3 polling. All other configuration fields are silently ignored during runtime S3 polling and require a server restart to take effect.

S3 Polling Security

• ETag-based polling — HEAD request checks for changes before full GET
• Fail-open on polling errors — If S3 becomes unreachable during operation, server continues with current config
• Fail-fast on startup — If S3 is unreachable at startup when configured, server fails to start
• No caching of stale config — Config is only updated when successfully fetched and parsed

Base64 Encoding for Keys

All private keys and certificates in S3 config must be base64-encoded:

• Standard base64 or URL-safe base64 accepted
• Prevents JSON parsing issues with PEM format
• Decoded and validated before use

Backup and Recovery

What to Back Up

Backup Strategy

Database

SQLite:

# Simple file copy (stop writes first or use backup API)
cp /data/vouch.db /backup/vouch.db.$(date +%Y%m%d_%H%M%S)

# Or use SQLite backup command (safe during writes)
sqlite3 /data/vouch.db ".backup '/backup/vouch.db.backup'"

PostgreSQL:

pg_dump -Fc vouch > /backup/vouch.$(date +%Y%m%d_%H%M%S).dump

Frequency: Daily minimum. More frequent for high-activity deployments.

Cryptographic Keys

Back up all keys to a secure, offline location:

# SSH CA key
cp ssh_ca_key /secure-backup/ssh_ca_key

# OIDC signing key (ES256)
cp oidc_signing_key.pem /secure-backup/oidc_signing_key.pem

# OIDC RSA signing key (RS256) — if configured
cp oidc_rsa_key.pem /secure-backup/oidc_rsa_key.pem

Store key backups:

• Encrypted at rest
• In a separate location from the server
• With restricted access (minimum two-person rule for production)

Recovery Procedures

Full Server Recovery

1. Deploy new server with the same configuration
2. Restore database from backup
3. Restore cryptographic keys (SSH CA, OIDC signing, JWT secret)
4. Start the server — migrations run automatically if needed
5. Verify: curl https://auth.example.com/health

Lost SSH CA Key

If the SSH CA key is lost and no backup exists:

1. Generate a new SSH CA key
2. Distribute the new public key to all SSH hosts
3. Configure Vouch with the new key
4. All users must run vouch login to get new certificates

Lost JWT Secret

If the JWT secret changes (lost or compromised):

1. Set the new VOUCH_JWT_SECRET
2. Restart the server
3. All existing sessions are invalidated
4. Users must run vouch login again

Database Corruption

1. Stop the server
2. Restore from backup
3. Users who enrolled after the backup will need to re-enroll
4. Start the server

Disaster Recovery Testing

Periodically test your recovery procedures:

1. Restore a database backup to a test environment
2. Start a test server with production keys
3. Verify enrollment, login, and credential flows
4. Document any issues and update procedures

Software Updates

Update Procedure

Pre-Update

1. Read the release notes for breaking changes
2. Back up the database before upgrading
3. Test in staging if possible

Server Update

# Back up database
cp /data/vouch.db /data/vouch.db.pre-upgrade

# Update via package manager
sudo apt upgrade vouch-server    # Debian/Ubuntu
sudo dnf upgrade vouch-server    # RHEL/Fedora

# Or via Docker
docker pull ghcr.io/vouch-sh/vouch:latest
docker compose up -d

# Or via Helm
helm upgrade vouch-server oci://ghcr.io/vouch-sh/charts/vouch-server \
  --version <new-version> --namespace vouch

# Verify
curl -k https://auth.example.com/health

Database migrations run automatically on startup. No manual migration steps are needed.

Client Update

# macOS
brew upgrade vouch

# Linux
sudo apt upgrade vouch    # Debian/Ubuntu
sudo dnf upgrade vouch    # RHEL/Fedora

Rollback

If an update causes issues:

1. Stop the server
2. Restore the database from pre-upgrade backup
3. Install the previous version
4. Start the server

Note: Database migrations may not be reversible. Always back up before upgrading.

Version Compatibility

• The server is backward-compatible with older CLI versions
• Upgrade the server first, then clients
• Major version bumps may require simultaneous client updates (documented in release notes)

Release Channels

Troubleshooting

Common Issues

Server Connection Issues

“Connection refused” or timeouts

1. Check server health: curl -k https://auth.example.com/health
2. Check DNS resolution: dig auth.example.com
3. Check TLS: openssl s_client -connect auth.example.com:443
4. Check firewall rules (port 443 must be accessible)

SCIM Provisioning Issues

User not de-provisioned

• Verify the SCIM bearer token is valid and not expired
• Check the SCIM audit log for errors
• Confirm the IdP is sending DELETE requests to the correct endpoint

SCIM token rejected

• Tokens are shown once at creation and cannot be retrieved after
• Generate a new token via the admin API (POST /api/v1/org/scim-tokens) and update the IdP configuration

Identity Provider Issues

“Failed to fetch upstream OIDC discovery document”

1. Verify the configured VOUCH_IDP_<SLUG>_ISSUER is correct and reachable: curl -s $VOUCH_IDP_<SLUG>_ISSUER/.well-known/openid-configuration | jq .issuer
2. Check that the issuer URL uses HTTPS (HTTP is only allowed for localhost)
3. Ensure the server can make outbound HTTPS requests (check firewall/proxy)

“Issuer mismatch” during OIDC discovery

• The issuer field in the discovery document must exactly match VOUCH_IDP_<SLUG>_ISSUER (trailing slashes matter)
• Some providers require a trailing slash (e.g., Auth0: https://tenant.auth0.com/)
• Entra /organizations/v2.0 is special-cased — its {tenantid} template issuer is accepted
• Entra /common/v2.0 is rejected at startup; use /organizations/v2.0 or a single-tenant URL (see Microsoft Entra ID)

“Failed to fetch SAML IdP metadata”

• Verify the configured VOUCH_IDP_<SLUG>_METADATA_URL is correct and reachable
• Check that the URL returns XML, not an HTML login page
• Ensure the server can make outbound HTTPS requests

SAML signature verification errors

• Confirm the IdP’s signing certificate in the metadata is current and not expired
• Ensure the server clock is synchronized via NTP — SAML assertions have time-based validity windows (typically 5 minutes of skew tolerance)
• Check the IdP assertion signing algorithm matches what the server expects

Enrollment broken after upgrade, and only the old VOUCH_OIDC_* / VOUCH_SAML_* env vars or S3 oidc / saml blocks are set

• The flat legacy variables and S3 blocks are no longer read by the server — they are silently ignored. Migrate to the unified VOUCH_IDPS + VOUCH_IDP_<SLUG>_* env vars (or the S3 idps array). See IdP Overview for the full mapping.

“Duplicate IdP slug”

• Every entry in VOUCH_IDPS / idps[].id must be unique. Rename one of them.

Debug Logging

Enable verbose logging for troubleshooting:

# Server
RUST_LOG=debug vouch-server

For component-specific logging:

RUST_LOG=vouch_server=debug

Getting Help

• GitHub Issues — Bug reports
• GitHub Discussions — Questions
• Security Issues — Security vulnerabilities

Incident Response

This chapter describes Vouch’s incident severity classification, response procedures, and communication channels for security events.

Severity Levels

Response Procedure

1. Triage — Assess severity and scope
2. Contain — Revoke affected credentials, disable vulnerable features
3. Investigate — Root cause analysis
4. Remediate — Deploy fix
5. Communicate — Notify affected users
6. Review — Post-incident analysis

Communication Channels

• Security advisories: https://vouch.sh/security
• CVE assignments: Via GitHub Security Advisories
• Status page: https://status.vouch.sh

Air-Gapped Deployment

This chapter covers deploying Vouch in environments with no internet connectivity, such as defense contractors, government agencies, financial services, and critical infrastructure.

Supported with operational constraints — You can run vouch-server and the vouch CLI on an isolated network today using the same binaries and deployment paths as on-premise (systemd, Docker, or Kubernetes). This chapter documents that workflow. A few operator conveniences (listed under Roadmap below) are not built into the product yet.

Supported today

Enrollment uses the standard vouch enroll device flow (browser opens the verification URL on your internal Vouch host) or browser-only /enroll/start on the server UI. There is no separate air-gap-only CLI mode.

For general on-prem deployment (reachable IdP, standard updates), start with Deployment Overview.

Roadmap

These items are not available in the product today; the chapters above describe manual procedures instead:

• Server syslog / SIEM streaming — use periodic database export in Operations until built-in export exists
• Headless enrollment — enrollment without any browser on the internal network

Overview

In an air-gapped environment:

• No SaaS services available
• Updates delivered via sneakernet
• Internal identity provider (no Google Workspace)
• Time sync from isolated NTP or GPS

Vouch’s built-in SSH CA and local-first architecture make it well-suited for these constraints.

Architecture

+--------------------------------------------------------------------------+
|                          AIR-GAPPED ENCLAVE                              |
|                                                                          |
|  +--------------------------------------------------------------------+  |
|  |                     On-Premises Vouch Stack                        |  |
|  |                                                                    |  |
|  |  +--------------+  +----------------+  +-----------------------+   |  |
|  |  |   Vouch      |  |   Built-in     |  |       SQLite          |   |  |
|  |  |   Server     |  |   SSH CA       |  |                       |   |  |
|  |  |              |  |                |  |  * Users & credentials |   |  |
|  |  |  * WebAuthn  |  |  * Ed25519 CA  |  |  * Sessions           |   |  |
|  |  |  * OIDC      |  |  * SSH certs   |  |  * Audit logs         |   |  |
|  |  |  * Sessions  |  |  * 8hr TTL     |  |                       |   |  |
|  |  +--------------+  +----------------+  +-----------------------+   |  |
|  |         |                  |                      |                |  |
|  |         +------------------+----------------------+                |  |
|  |                            |                                       |  |
|  +----------------------------+---------------------------------------+  |
|                               |                                          |
|                               | Internal Network Only                    |
|                               v                                          |
|  +--------------------------------------------------------------------+  |
|  |                        Workstations                                |  |
|  |                                                                    |  |
|  |  +--------------+  +--------------+  +-------------------------+   |  |
|  |  | Workstation  |  | Workstation  |  |   Protected Resources   |   |  |
|  |  |              |  |              |  |                         |   |  |
|  |  | * vouch CLI  |  | * vouch CLI  |  |  * SSH servers          |   |  |
|  |  | * YubiKey    |  | * YubiKey    |  |  * Internal apps        |   |  |
|  |  | * Certs      |  | * Certs      |  |  * Databases            |   |  |
|  |  +--------------+  +--------------+  +-------------------------+   |  |
|  |                                                                    |  |
|  +--------------------------------------------------------------------+  |
|                                                                          |
|  +--------------------------------------------------------------------+  |
|  |                       Time Infrastructure                          |  |
|  |  +------------+     +-----------------+                            |  |
|  |  | GPS Time   |---->|  Internal NTP   |----> All hosts             |  |
|  |  | Receiver   |     |  (stratum 1)    |                            |  |
|  |  +------------+     +-----------------+                            |  |
|  +--------------------------------------------------------------------+  |
+--------------------------------------------------------------------------+
                                    |
                                    | Air Gap (sneakernet)
                                    v
+--------------------------------------------------------------------------+
|                         CONNECTED ENVIRONMENT                            |
|                                                                          |
|  * Signed software packages (from packages.vouch.sh)                     |
|  * CA certificate updates                                                |
|  * (Optional) Audit log export                                           |
+--------------------------------------------------------------------------+

Identity Provider Considerations

In an air-gapped environment, you cannot use external identity providers like Google Workspace for enrollment. Vouch Server requires at least one upstream IdP to verify user identity, so an air-gapped deployment must include a self-hosted IdP inside the enclave:

• Self-hosted OIDC provider — Deploy an internal OIDC-compliant IdP inside the enclave (e.g., Keycloak, Dex, or Microsoft AD FS). Add it to Vouch Server’s VOUCH_IDPS list with VOUCH_IDP_<SLUG>_TYPE=oidc plus the _ISSUER, _CLIENT_ID, and _CLIENT_SECRET variables pointing to the internal IdP.
• Self-hosted SAML provider — Deploy an internal SAML IdP (e.g., Shibboleth, AD FS) and configure it with VOUCH_IDP_<SLUG>_TYPE=saml plus VOUCH_IDP_<SLUG>_METADATA_URL pointing to the internal metadata document.

Prerequisites

Hardware

• Servers for Vouch stack (VMs or bare metal)
• YubiKey 5 series for each user (firmware 5.2+)
• GPS receiver for time sync (recommended)
• USB drives for sneakernet transfers

Software (Pre-downloaded)

• Vouch Server packages (RPM/DEB from packages.vouch.sh)
• vouch CLI packages (RPM/DEB from packages.vouch.sh)
• Container images and/or Helm charts (for Kubernetes deployments)

Installation

This chapter walks through the complete installation procedure for deploying Vouch in an air-gapped environment, from downloading packages on a connected machine through enrolling users on the isolated network.

Step 1: Download Packages for Offline Transfer

On a connected machine, download the required packages from packages.vouch.sh:

# Import Vouch GPG signing key
curl -fsSL https://packages.vouch.sh/gpg/vouch.asc | gpg --import

# Download server RPM
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-server-1.0.0-1.x86_64.rpm

# Download CLI RPM (for each workstation architecture)
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-1.0.0-1.x86_64.rpm
curl -LO https://packages.vouch.sh/rpm/aarch64/vouch-1.0.0-1.aarch64.rpm

# For Debian/Ubuntu workstations
curl -LO https://packages.vouch.sh/apt/vouch_1.0.0_amd64.deb
curl -LO https://packages.vouch.sh/apt/vouch_1.0.0_arm64.deb

For container-based or Kubernetes deployments, also download:

# Pull and save container image
docker pull ghcr.io/vouch-sh/vouch:1.0.0
docker save ghcr.io/vouch-sh/vouch:1.0.0 -o vouch-server-1.0.0.tar

# Download Helm chart (for Kubernetes)
helm pull oci://ghcr.io/vouch-sh/charts/vouch-server --version 0.1.0

Generate checksums for verification after transfer:

sha256sum vouch-server-*.rpm vouch-*.rpm vouch-*.deb vouch-server-*.tar > SHA256SUMS
gpg --detach-sign SHA256SUMS

Transfer all files to the air-gapped environment via approved media.

Step 2: Verify Package Integrity

On the air-gapped network:

# Import Vouch GPG signing key (transferred separately, verified out-of-band)
gpg --import vouch-release-key.pub

# Verify checksums
gpg --verify SHA256SUMS.sig SHA256SUMS
sha256sum -c SHA256SUMS

# Verify RPM signatures
rpm -K vouch-server-1.0.0-1.x86_64.rpm
rpm -K vouch-1.0.0-1.x86_64.rpm

Step 3: Install Packages

RPM-based installation (recommended for bare metal/VM):

# Install server
rpm -ivh vouch-server-1.0.0-1.x86_64.rpm

# Install CLI on workstations
rpm -ivh vouch-1.0.0-1.x86_64.rpm

DEB-based installation:

# Install CLI on Debian/Ubuntu workstations
dpkg -i vouch_1.0.0_amd64.deb

Container-based installation:

# Load container image into local Docker registry
docker load < vouch-server-1.0.0.tar

# Verify image loaded
docker images | grep vouch

Step 4: Secure Key Generation

This is a critical security operation. Follow your organization’s key ceremony procedures.

Vouch requires several cryptographic keys for different purposes. All key generation should be performed on a trusted, air-gapped workstation. See the Key Ceremony chapter for detailed instructions on generating each key.

Key Overview

*Auto-generates ephemeral key if not provided (not recommended for production).

Note: All PEM-formatted keys and certificates must be base64-encoded when passed via environment variables. This ensures proper handling of newlines and special characters.

Step 5: Database Setup

Vouch uses SQLite by default, which is suitable for single-node deployments. The database is created automatically on first startup.

# SQLite (default, single-node)
export VOUCH_DATABASE_URL="sqlite:/data/vouch.db?mode=rwc"

# Create data directory with appropriate permissions
mkdir -p /data
chmod 700 /data

For high-availability deployments, a local PostgreSQL instance is supported:

# PostgreSQL (multi-node, must be reachable on the internal network)
export VOUCH_DATABASE_URL="postgres://user:password@db.internal:5432/vouch"

Database migrations run automatically on server startup.

Step 6: Configure Vouch Server

Vouch is configured entirely through environment variables. Create a secure environment file:

# Create environment file (chmod 600 after editing)
cat > /etc/vouch/vouch.env << 'EOF'
# =============================================================================
# Vouch Server Configuration - Air-Gapped Environment
# =============================================================================

# -----------------------------------------------------------------------------
# Required Configuration
# -----------------------------------------------------------------------------

# JWT signing secret (minimum 32 characters)
VOUCH_JWT_SECRET=<your-64-character-secret-here>

# Relying Party configuration
VOUCH_RP_ID=auth.internal
VOUCH_RP_NAME=Vouch (Air-Gapped)

# Database
VOUCH_DATABASE_URL=sqlite:/data/vouch.db?mode=rwc

# -----------------------------------------------------------------------------
# Network Configuration
#
# Development (no TLS):
#   Server listens on VOUCH_LISTEN_ADDR (default: 0.0.0.0:3000)
#
# Production (TLS enabled):
#   Server automatically listens on port 443 (HTTPS) and port 80 (HTTP redirect)
#   VOUCH_LISTEN_ADDR is ignored when TLS is configured
#   HTTP requests on port 80 are 308 redirected to HTTPS on port 443
#   The /health endpoint is accessible on HTTP (for load balancer health checks)
#   Host header is validated against rp_id to prevent injection attacks
#   Requires CAP_NET_BIND_SERVICE capability (handled by packaging scripts)
# -----------------------------------------------------------------------------

# Listen address (used only when TLS is NOT configured)
VOUCH_LISTEN_ADDR=0.0.0.0:3000

# Base URL (how clients reach the server)
VOUCH_BASE_URL=https://auth.internal

# -----------------------------------------------------------------------------
# TLS Configuration (base64-encoded PEM)
# Generate with: base64 -i cert.pem | tr -d '\n'
# -----------------------------------------------------------------------------

VOUCH_TLS_CERT=<base64-encoded-certificate>
VOUCH_TLS_KEY=<base64-encoded-private-key>

# -----------------------------------------------------------------------------
# SSH CA Configuration (base64-encoded PEM)
# Generate with: base64 -i ssh_ca_key | tr -d '\n'
# -----------------------------------------------------------------------------

# SSH CA private key (base64-encoded PEM, takes precedence over path)
VOUCH_SSH_CA_KEY=<base64-encoded-ssh-ca-private-key>

# Or use a file path instead (file contains raw PEM, not base64):
# VOUCH_SSH_CA_KEY_PATH=/secrets/ssh_ca_key

# -----------------------------------------------------------------------------
# OIDC Provider Configuration
# Generate with: base64 -i oidc_signing_key.pem | tr -d '\n'
# -----------------------------------------------------------------------------

# Vouch acts as an OIDC provider - this key signs the ID tokens (base64-encoded PEM)
VOUCH_OIDC_SIGNING_KEY=<base64-encoded-oidc-signing-key>

# -----------------------------------------------------------------------------
# Upstream Identity Provider (Required)
# At least one IdP must be configured; the server refuses to start without one.
# -----------------------------------------------------------------------------

VOUCH_IDPS=internal-oidc
VOUCH_IDP_INTERNAL_OIDC_TYPE=oidc
VOUCH_IDP_INTERNAL_OIDC_ISSUER=https://idp.internal
VOUCH_IDP_INTERNAL_OIDC_CLIENT_ID=vouch-client
VOUCH_IDP_INTERNAL_OIDC_CLIENT_SECRET=<client-secret>

# -----------------------------------------------------------------------------
# Session Configuration
# -----------------------------------------------------------------------------

# Session duration (default: 8 hours)
VOUCH_SESSION_HOURS=8

# Device code settings (for CLI enrollment)
VOUCH_DEVICE_CODE_EXPIRES=600
VOUCH_DEVICE_POLL_INTERVAL=5

# -----------------------------------------------------------------------------
# Security Configuration
# -----------------------------------------------------------------------------

# Allowed email domains for enrollment (comma-separated)
VOUCH_ALLOWED_DOMAINS=internal,company.local

# DPoP (Demonstrating Proof of Possession)
VOUCH_DPOP_ENABLED=true
VOUCH_DPOP_NONCE_REQUIRED=false
VOUCH_DPOP_MAX_AGE=300

# -----------------------------------------------------------------------------
# Audit and Retention
# -----------------------------------------------------------------------------

# Cleanup interval (minutes, 0 to disable)
VOUCH_CLEANUP_INTERVAL=15

# Event retention (days)
VOUCH_AUTH_EVENTS_RETENTION_DAYS=730
VOUCH_OAUTH_EVENTS_RETENTION_DAYS=90

# -----------------------------------------------------------------------------
# Branding (Optional)
# -----------------------------------------------------------------------------

VOUCH_ORG_NAME=Your Organization
EOF

# Secure the environment file
chmod 600 /etc/vouch/vouch.env

Environment Variables Reference

Step 7: Deploy Services

There are several options for deploying the Vouch server.

Option A: Systemd Service (RPM Install)

If you installed via RPM, the vouch-server systemd service is configured automatically:

# Configure environment
cp /etc/vouch/vouch.env /etc/vouch/vouch.env.local
# Edit /etc/vouch/vouch.env.local with your settings

# Start and enable the service
systemctl enable --now vouch-server

# Check status
systemctl status vouch-server

# View logs
journalctl -u vouch-server -f

Option B: Docker Compose

# docker-compose.yml
services:
  vouch-server:
    image: ghcr.io/vouch-sh/vouch:1.0.0
    container_name: vouch-server
    restart: unless-stopped
    ports:
      - "443:443"
    volumes:
      - vouch-data:/data
      - /etc/vouch/secrets:/secrets:ro
    env_file:
      - /etc/vouch/vouch.env
    environment:
      # Override or add environment variables here
      VOUCH_DATABASE_URL: sqlite:/data/vouch.db?mode=rwc
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  vouch-data:

# Start services
docker-compose up -d

# Verify container is running
docker-compose ps

# Check logs for startup errors
docker-compose logs -f vouch-server

Option C: Helm Chart (Kubernetes)

A Helm chart is available for Kubernetes deployments. After transferring the chart archive to the air-gapped environment:

# Install from the downloaded chart archive
helm install vouch-server vouch-server-0.1.0.tgz \
  --namespace vouch \
  --create-namespace \
  --set image.repository=vouch-server \
  --set image.tag=1.0.0 \
  --values my-values.yaml

See the chart’s values.yaml for all configurable options including secrets, ingress, and persistent storage.

Verify Deployment

Regardless of deployment method:

# Verify health endpoint
curl -k https://auth.internal/health
# Expected: {"status":"healthy"}

# Verify SSH CA is loaded (if configured)
curl -k https://auth.internal/v1/credentials/ssh/ca
# Expected: ssh-ed25519 AAAA... vouch-ca@auth.internal

Step 8: Distribute CA Public Key

The SSH CA public key must be trusted by all SSH servers in the air-gapped environment:

# Fetch CA public key via API
curl -k https://auth.internal/v1/credentials/ssh/ca > vouch-ca.pub

# Copy to all SSH servers
scp vouch-ca.pub root@server:/etc/ssh/vouch-ca.pub

# Configure SSH server to trust the CA
echo "TrustedUserCAKeys /etc/ssh/vouch-ca.pub" >> /etc/ssh/sshd_config

# Optionally, configure authorized principals
echo "AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u" >> /etc/ssh/sshd_config

# Restart SSH daemon
systemctl restart sshd

AuthorizedPrincipals Setup (Optional but Recommended):

# Create principals directory
mkdir -p /etc/ssh/auth_principals

# For each user, create a file with allowed principals
# Vouch issues certificates with two principals: email and username
echo "john@company.internal" > /etc/ssh/auth_principals/john
echo "john" >> /etc/ssh/auth_principals/john

Step 9: Configure CLI for Air-Gap

# ~/.config/vouch/config.json
{
  "server_url": "https://auth.internal",
  "ca_cert_path": "/etc/vouch/root-ca.crt"
}

Or via environment:

export VOUCH_SERVER=https://auth.internal
export VOUCH_CA_CERT=/etc/vouch/root-ca.crt

Step 10: Enroll Users

Important: Enrollment requires browser access to the Vouch server’s web UI on the internal network. Users run vouch enroll on a workstation that can open the device verification URL against the internal Vouch host; there is no separate headless-only enroll mode.

Each user:

1. Opens a browser to https://auth.internal/enroll
2. Authenticates via the configured identity provider
3. Registers their YubiKey through the browser’s WebAuthn prompt (touch + PIN)

After enrollment, daily login uses the CLI (vouch login) with no browser required.

Key Ceremony

This chapter details the cryptographic key generation procedures required for an air-gapped Vouch deployment. All key generation should be performed on a trusted, air-gapped workstation following your organization’s key ceremony procedures.

JWT Secret Generation (Required)

The JWT secret is used to sign all OAuth tokens and session cookies. It must be at least 32 characters.

# Generate cryptographically secure 64-character secret
openssl rand -base64 48

# Alternative using /dev/urandom
head -c 48 /dev/urandom | base64

# Store securely - this will be VOUCH_JWT_SECRET

Security Notes:

• Use a minimum of 32 characters (48+ recommended)
• Never reuse secrets across environments
• Rotate periodically (requires re-authentication of all users)

SSH CA Key Generation (Ed25519)

The SSH CA signs user SSH certificates. If not provided, SSH certificate issuance will be disabled.

# Generate Ed25519 SSH CA key pair (no passphrase for automated use)
ssh-keygen -t ed25519 -f ssh_ca_key -N "" -C "vouch-ca@auth.internal"

# Set restrictive permissions
chmod 600 ssh_ca_key

# Verify key type and fingerprint
ssh-keygen -l -f ssh_ca_key
# Output: 256 SHA256:xxxx vouch-ca@auth.internal (ED25519)

# View public key (for distribution to SSH servers)
cat ssh_ca_key.pub

Environment variable format (base64-encoded):

# Option 1: Provide base64-encoded key content (preferred for containers)
export VOUCH_SSH_CA_KEY="$(base64 -i ssh_ca_key | tr -d '\n')"

# Option 2: Provide path to key file (file contains raw PEM, not base64)
export VOUCH_SSH_CA_KEY_PATH="/secrets/ssh_ca_key"

Key Storage Options:

• HSM (recommended for high-security) – Store in hardware security module
• Encrypted file with split knowledge – Two administrators hold partial keys
• YubiKey PIV (for smaller deployments) – Store on hardware token

OIDC Signing Key Generation (P-256 ECDSA)

The OIDC signing key signs ID tokens using ES256 algorithm. If not provided, an ephemeral key is generated on each server restart (not recommended for production as it invalidates all existing tokens).

# Generate P-256 EC private key in PKCS#8 format
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:prime256v1 -out oidc_signing_key.pem

# Set restrictive permissions
chmod 600 oidc_signing_key.pem

# Verify key type
openssl ec -in oidc_signing_key.pem -text -noout 2>/dev/null | head -3
# Output should include: Private-Key: (256 bit, prime256v1)

# Extract public key (for debugging/verification)
openssl ec -in oidc_signing_key.pem -pubout -out oidc_signing_key.pub

Environment variable format (base64-encoded):

# Provide base64-encoded PEM content
export VOUCH_OIDC_SIGNING_KEY="$(base64 -i oidc_signing_key.pem | tr -d '\n')"

OIDC RSA Signing Key Generation (RSA-3072)

The OIDC RSA signing key signs ID tokens with RS256 algorithm per OIDC Core Section 3.1.3.7 and all AWS credential tokens (/v1/credentials/aws/token, serving both STS AssumeRoleWithWebIdentity and IAM Identity Center CreateTokenWithIAM). Any deployment using the AWS integration must provide a durable key — without one, an ephemeral RSA-3072 key is generated on each server restart, and AWS token verification breaks after restarts and across multiple instances.

# Generate RSA-3072 private key in PKCS#8 format
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out oidc_rsa_key.pem

# Set restrictive permissions
chmod 600 oidc_rsa_key.pem

# Verify key type and size
openssl rsa -in oidc_rsa_key.pem -text -noout 2>/dev/null | head -3
# Output should include: Private-Key: (3072 bit)

Environment variable format (base64-encoded):

# Provide base64-encoded PEM content
export VOUCH_OIDC_RSA_SIGNING_KEY="$(base64 -i oidc_rsa_key.pem | tr -d '\n')"

TLS Certificate Generation

For production, use certificates signed by your internal CA. For testing, self-signed certificates can be used.

# Generate EC private key and self-signed certificate
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
  -keyout tls_key.pem -out tls_cert.pem -days 365 -nodes \
  -subj "/CN=auth.internal" \
  -addext "subjectAltName=DNS:auth.internal,DNS:localhost"

# Set restrictive permissions
chmod 600 tls_key.pem

# Verify certificate
openssl x509 -in tls_cert.pem -text -noout | head -15

Environment variable format (base64-encoded):

# Base64 encode for environment variables
export VOUCH_TLS_CERT="$(base64 -i tls_cert.pem | tr -d '\n')"
export VOUCH_TLS_KEY="$(base64 -i tls_key.pem | tr -d '\n')"

Key Security Best Practices

1. File Permissions: Always use chmod 600 for private keys
2. Never Commit Keys: Add *.pem, *_key, *.key to .gitignore
3. Audit Key Access: Log all access to key material
4. Backup Securely: Store encrypted backups in separate secure location
5. Document Fingerprints: Record key fingerprints in secure documentation
6. Key Rotation: Plan for periodic rotation (SSH CA annually, JWT secret quarterly)

YubiKey Provisioning

In an air-gapped environment, YubiKey provisioning is done entirely on the internal network through the Vouch server’s web UI. This chapter covers the provisioning workflow, hardware requirements, and spare key strategy.

Provisioning Workflow

1. Administrator creates a user account via the Vouch server web interface
2. User navigates to https://auth.internal on their workstation browser
3. User inserts their YubiKey and completes the WebAuthn registration flow
4. User sets a PIN on their YubiKey if one is not already configured (minimum 8 characters)
5. The credential is registered and the user can begin authenticating

YubiKey Requirements

• YubiKey 5 series with firmware 5.2+
• FIDO2/WebAuthn support enabled
• PIN configured (minimum 8 characters)

Spare Key Strategy

Each user should register at least two YubiKeys (primary and backup). If a YubiKey is lost or damaged:

1. User reports lost key to administrator
2. Administrator revokes the lost key’s credential via the web UI
3. User registers their backup YubiKey

Operations

This chapter covers the day-to-day operational procedures for maintaining a Vouch deployment in an air-gapped environment, including time synchronization, software updates, audit log export, disaster recovery, and troubleshooting.

Time Synchronization

Certificate validity depends on accurate time. Options for air-gapped networks:

GPS Time Receiver (Recommended)

+----------------+     +--------------------+
| GPS Receiver   |---->| Internal NTP       |
| (one-way data) |     | Server (stratum 1) |
+----------------+     +--------------------+
         |                      |
         |                      v
    One-way only         All internal hosts
    (no data out)

Configure NTP clients:

# /etc/ntp.conf
server ntp.internal iburst

Manual Time Sync

For truly isolated networks without GPS:

1. Reference time from secure source (atomic clock, verified external)
2. Set time on NTP server manually
3. Document time sync in audit log

Vouch server configuration is done via environment variables (see Configure Vouch Server). JWT clock skew tolerance is handled automatically.

Software Updates

Update Procedure

1. Download updated packages (connected environment)

# Download latest packages from packages.vouch.sh
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-server-1.1.0-1.x86_64.rpm
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-1.1.0-1.x86_64.rpm

# For container deployments
docker pull ghcr.io/vouch-sh/vouch:1.1.0
docker save ghcr.io/vouch-sh/vouch:1.1.0 -o vouch-server-1.1.0.tar

2. Verify signatures (connected environment)

rpm -K vouch-server-1.1.0-1.x86_64.rpm
rpm -K vouch-1.1.0-1.x86_64.rpm

3. Transfer via approved media (sneakernet)

4. Verify again (air-gapped environment)

rpm -K vouch-server-1.1.0-1.x86_64.rpm
sha256sum -c SHA256SUMS

5. Apply update

For RPM installations:

# Backup database before upgrade
cp /data/vouch.db /data/vouch.db.backup.$(date +%Y%m%d)

# Upgrade package (migrations run automatically on next startup)
rpm -Uvh vouch-server-1.1.0-1.x86_64.rpm

# Restart service
systemctl restart vouch-server

# Verify health
curl -k https://auth.internal/health

For container deployments:

docker load < vouch-server-1.1.0.tar
# Update docker-compose.yml image tag, then:
docker-compose up -d

Rollback

For RPM installations:

# Restore database backup
cp /data/vouch.db.backup.YYYYMMDD /data/vouch.db

# Downgrade package
rpm -Uvh --oldpackage vouch-server-1.0.0-1.x86_64.rpm

# Restart service
systemctl restart vouch-server

Audit Log Export

Air-gapped environments still need audit trails for compliance.

One-Way Data Diode

+-----------------+     +-------------+     +-----------------+
| Air-Gapped      |---->| Data Diode  |---->| SIEM            |
| Vouch Server    |     | (hardware)  |     | (connected)     |
|                 |     |             |     |                 |
| UDP syslog out  |     | One-way     |     | Splunk/Datadog  |
+-----------------+     +-------------+     +-----------------+

Roadmap: Built-in syslog/SIEM forwarding from vouch-server is not implemented yet. Use the periodic export method below (or your database backup process — see Backup and Recovery).

Periodic Export

#!/bin/bash
# Weekly audit log export script

DATE=$(date +%Y%m%d)
OUTPUT_DIR=/mnt/export

# Export audit logs from SQLite directly
sqlite3 /data/vouch.db \
  ".mode json" \
  "SELECT * FROM auth_events WHERE created_at >= datetime('now', '-7 days');" \
  > $OUTPUT_DIR/audit-$DATE.json

# Encrypt for transport
gpg --encrypt --recipient auditor@company.com \
  $OUTPUT_DIR/audit-$DATE.json

# Generate checksum
sha256sum $OUTPUT_DIR/audit-$DATE.json.gpg > $OUTPUT_DIR/audit-$DATE.sha256

# Remove unencrypted
rm $OUTPUT_DIR/audit-$DATE.json

echo "Export complete: audit-$DATE.json.gpg"

Transfer encrypted exports via approved media to connected compliance systems.

Disaster Recovery

Backup Strategy

Recovery Procedure

1. Stop the service

systemctl stop vouch-server

2. Restore database from backup

cp /data/vouch.db.backup.YYYYMMDD /data/vouch.db
chown vouch:vouch /data/vouch.db

3. Re-sync time

# Verify NTP synchronization
timedatectl status
chronyc tracking  # or ntpq -p

4. Start and validate

systemctl start vouch-server
curl -k https://auth.internal/health

CA Key Recovery

If CA keys are lost, all issued certificates become unverifiable.

Prevention:

• Store CA keys in HSM with backup
• Use split-knowledge for key recovery
• Document key ceremony procedures

Recovery:

1. Generate new CA from backup
2. Re-provision all user credentials
3. Redistribute new CA public key
4. Update all SSH server trust anchors

Security Considerations

Network Segmentation

+-------------------------------------------------------------+
|                    Air-Gapped Network                        |
|                                                              |
|  +-----------------+        +-----------------------------+  |
|  |   Management    |        |      User Network           |  |
|  |   VLAN          |        |                             |  |
|  |                 |        |  +-------+  +-----------+   |  |
|  |  * Vouch Server |<------>|  |Workst.|  | Protected |   |  |
|  |  * SQLite       |        |  +-------+  | Resources |   |  |
|  |                 |        |             +-----------+   |  |
|  +-----------------+        +-----------------------------+  |
|           |                                                  |
|           | Restricted                                       |
|           v                                                  |
|  +-----------------+                                         |
|  | Admin Jumpbox   | <-- Physical access control             |
|  +-----------------+                                         |
+--------------------------------------------------------------+

Physical Security

• Server room access controls
• YubiKey storage procedures
• Media transfer protocols
• Tamper-evident logging

Compliance Mapping

Troubleshooting

Cannot Connect to Vouch Server

# Check network connectivity
ping auth.internal

# Verify TLS
openssl s_client -connect auth.internal:443 -CAfile /etc/vouch/root-ca.crt

# Check server logs (systemd)
journalctl -u vouch-server --since "1 hour ago"

# Check server logs (Docker)
docker-compose logs vouch-server

Certificate Validation Failures

# Check system time
date
timedatectl status

# Verify CA is trusted
ssh-keygen -L -f /path/to/cert  # View certificate details

# Check certificate dates
ssh-keygen -L -f /path/to/cert | grep Valid

YubiKey Not Recognized

# Check USB connection
lsusb | grep Yubico

# Verify FIDO2 functionality
ykman fido info

# Reset FIDO2 application (destructive - re-enrollment required)
ykman fido reset

Environment Variables

All Vouch server configuration is done via environment variables (prefixed with VOUCH_). These can also be passed as command-line arguments.

Core Configuration

Network

Upstream Identity Provider

Configure one or more upstream IdPs (OIDC, SAML, or any mix) as a single unified list. VOUCH_IDPS holds a comma-separated list of slugs; each slug picks up its VOUCH_IDP_<SLUG>_* variables. Slugs match [a-z0-9-]{1,32} (no leading or trailing hyphen) and must be unique.

Hyphens in slugs become underscores in variable names: a slug of corp-saml becomes VOUCH_IDP_CORP_SAML_*.

OIDC IdP (per slug)

OIDC IdPs auto-discover authorization, token, and JWKS endpoints from {issuer}/.well-known/openid-configuration at startup.

SAML IdP (per slug)

Legacy variables (no longer read)

The previous flat single-IdP variables — VOUCH_OIDC_ISSUER, VOUCH_OIDC_CLIENT_ID, VOUCH_OIDC_CLIENT_SECRET, VOUCH_OIDC_PROVIDERS, VOUCH_SAML_IDP_METADATA_URL, VOUCH_SAML_SP_ENTITY_ID, VOUCH_SAML_EMAIL_ATTRIBUTE, VOUCH_SAML_DOMAIN_ATTRIBUTE — are silently ignored. Setting them configures nothing; only the unified VOUCH_IDPS / VOUCH_IDP_<SLUG>_* variables above are read. See IdP Overview for the field-by-field mapping.

Session

SSH CA

OIDC Signing

AWS KMS

DPoP

Cleanup & Retention

CORS

GitHub App

These variables configure the Vouch GitHub App integration for issuing GitHub tokens. The App ID, name, and key are required together for GitHub App functionality. OAuth client ID and secret are additionally needed for GitHub user authentication.

TLS

When both VOUCH_TLS_CERT and VOUCH_TLS_KEY are set, the server listens on port 443 (HTTPS) with an automatic HTTP-to-HTTPS redirect on port 80. The VOUCH_LISTEN_ADDR setting is ignored when TLS is configured.

S3 Configuration

Vouch supports loading configuration from an S3 object for centralized management. S3 configuration values override environment variables.

JWT Assertion

Protected Resource Metadata (RFC 9728)

These optional variables configure descriptive metadata published in the OAuth 2.0 Protected Resource Metadata document at /.well-known/oauth-protected-resource.

CLI Download URLs

These optional variables configure download links displayed in the server UI.

CLI Localization

The Vouch CLI resolves its user-facing language once at startup. Resolution checks the following sources in order; the first that parses as a BCP 47 language tag wins.

If none of the above resolve, the CLI falls through to the operating system’s default locale via sys-locale. When that is also unavailable (e.g., a minimal container with no locale configured), the CLI falls back to en-US. POSIX-style locale strings (en_US.UTF-8, ca_ES@valencia) are normalized by stripping the .<codeset> and @<modifier> suffixes and replacing _ with -.

Machine-readable output (vouch status --json, vouch status --shell, vouch env, credential helpers) is never affected by the locale — those payloads are consumed by other tools and remain stable across locales. Only human-facing prompts, success/failure messages, error messages, and --help text honor the negotiated locale.

This release ships en-US only; the catalog scaffolding lets translations land in isolated PRs without changing the CLI surface.

File Locations (CLI & Agent)

The vouch CLI and vouch-agent follow the XDG Base Directory Specification. Files are split by purpose across the standard base directories, and the relevant XDG_* environment variables are honored on all platforms — including macOS, where Vouch uses ~/.config rather than ~/Library/Application Support, matching the convention of tools like gh and git.

Locations

All files written by Vouch are created with owner-only permissions (0600 for files, 0700 for directories) on Unix.

Runtime sockets

Unix sockets live in XDG_RUNTIME_DIR — a private, user-owned, 0700 tmpfs that the login session manager clears on logout. XDG_RUNTIME_DIR is only set on Linux sessions managed by a login manager; on macOS and headless logins it is absent, so Vouch falls back to a 0700 directory under the cache base (~/.cache/vouch), chosen because its short path stays within the macOS sun_path socket-path length limit.

Migration from ~/.vouch/

Earlier versions of Vouch stored everything flat under ~/.vouch/. On the first run of a newer vouch or vouch-agent, files found there (config.json, cookie.txt, audit.log, client_key.json) are moved automatically to their XDG locations, preserving permissions. The migration is idempotent and prints a one-time notice. Sockets are not migrated — they are recreated on demand.

On macOS (and Windows), the agent’s agent.pid/agent.log previously lived in the OS cache directory (~/Library/Caches/vouch) rather than ~/.cache/vouch. These are migrated too, so an upgraded agent detects a still-running old agent via the PID file instead of starting a second one. On Linux the cache location is unchanged, so this step is a no-op.

One thing the migration cannot fix automatically: if you previously ran vouch setup ssh, your ~/.ssh/config may contain an IdentityAgent line pointing at the old ~/.vouch/ssh-agent.sock. Re-running vouch setup ssh detects and rewrites that stale path to the new socket location.

S3 Configuration Schema

For production deployments, Vouch supports loading configuration from an S3 object. This enables:

• Centralized management — Single source of truth for multi-instance deployments
• Dynamic updates — Configuration changes without server restart (for supported fields)
• TLS hot-reload — Automatic certificate rotation without downtime
• Secrets management — Leverage S3 server-side encryption and IAM for credential protection

Enabling S3 Configuration

Set the following environment variables to enable S3-based configuration:

# Required: bucket name
VOUCH_S3_CONFIG_BUCKET=my-bucket

# Optional: object key (default: config/vouch-server.json)
VOUCH_S3_CONFIG_KEY=config/vouch-server.json

# Optional: AWS region (uses default credential chain region if not set)
VOUCH_S3_CONFIG_REGION=us-west-2

# Optional: polling interval in seconds (default: 60)
VOUCH_S3_CONFIG_POLL_INTERVAL=60

When S3 configuration is enabled, it overrides environment variables. This allows for centralized configuration management with dynamic updates.

JSON Schema

The S3 configuration file is a JSON document with the following schema:

{
  "version": 1,
  "listen_addr": "0.0.0.0:443",
  "rp_id": "vouch.example.com",
  "rp_name": "Example Corp",
  "base_url": "https://vouch.example.com",
  "database_url": "postgres://...",
  "dsql_endpoints": {
    "us-east-1": "postgres://vouch@abc123.dsql.us-east-1.on.aws/postgres"
  },
  "jwt_secret": "32+ character secret",
  "session_hours": 8,
  "org_name": "Example Corp",
  "tls": {
    "cert": "<base64-encoded PEM certificate>",
    "key": "<base64-encoded PEM private key>"
  },
  "idps": [
    {
      "id": "google",
      "type": "oidc",
      "issuer": "https://accounts.google.com",
      "client_id": "...",
      "client_secret": "..."
    },
    {
      "id": "corp-saml",
      "type": "saml",
      "metadata_url": "https://idp.example.com/saml/metadata",
      "sp_entity_id": "https://vouch.example.com",
      "email_attribute": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
      "domain_attribute": "department"
    }
  ],
  "allowed_domains": ["example.com"],
  "ssh_ca_key": "<base64-encoded PEM Ed25519 private key>",
  "ssh_ca_kms_key_id": "mrk-1234abcd5678efgh",
  "oidc_signing_key": "<base64-encoded PEM EC P-256 private key>",
  "oidc_signing_kms_key_id": "mrk-abcd1234efgh5678",
  "oidc_rsa_signing_key": "<base64-encoded PEM RSA-3072 private key>",
  "oidc_rsa_signing_kms_key_id": "mrk-rsa1234abcd5678",
  "jwt_hmac_kms_key_id": "mrk-5678abcd1234efgh",
  "document_key": {
    "kms_key_id": "mrk-<key-id>",
    "encrypted_private_key": "<base64-encoded KMS ciphertext>",
    "algorithm": "p384"
  },
  "dpop": {
    "max_age_seconds": 300
  },
  "cors_origins": ["https://app.example.com"],
  "github": {
    "app_id": 12345,
    "app_name": "my-vouch-app",
    "app_key": "<PEM RSA private key>",
    "webhook_secret": "<secret>",
    "client_id": "<oauth-client-id>",
    "client_secret": "<oauth-client-secret>"
  },
  "cleanup_interval_minutes": 15,
  "auth_events_retention_days": 90,
  "oauth_events_retention_days": 90,
  "resource_name": "Vouch",
  "resource_documentation": "https://vouch.sh/docs/",
  "resource_policy_uri": "https://vouch.sh/privacy/",
  "resource_tos_uri": "https://vouch.sh/terms/",
  "cli_download_macos": "https://example.com/vouch-macos",
  "cli_download_linux": "https://example.com/vouch-linux",
  "cli_download_windows": "https://example.com/vouch-windows",
  "device_code_expires_seconds": 600,
  "device_poll_interval_seconds": 5
}

Field Descriptions

Base64 Encoding

All certificate and key fields in the S3 configuration must be base64-encoded PEM strings. To encode a PEM file:

# Encode a PEM file for S3 config
base64 -i cert.pem | tr -d '\n'

This ensures proper handling of newlines and special characters within JSON values.

Hot-Reloadable vs Startup-Only Fields

The server polls S3 at the configured interval and detects changes via ETag comparison. However, only certain fields support hot-reload without a server restart:

Non-hot-reloadable fields include: jwt_secret, database_url, listen_addr, rp_id, rp_name, session_hours, cors_origins, allowed_domains, dpop.*, idps[], GitHub App settings, SSH CA key, OIDC signing keys, and all KMS key IDs.

Legacy blocks silently ignored: The previous top-level oidc and saml blocks (single-IdP nested objects) are no longer read by the server. They configure nothing; only entries inside the idps array are honored. See IdP Overview for the field-by-field mapping.

Changes to non-hot-reloadable fields in S3 are silently ignored. A server restart is required to apply them.

TLS Certificate Hot-Reload

Vouch supports automatic TLS certificate reloading without dropping connections:

1. Via S3 polling — Update tls.cert and tls.key in the S3 config; the server detects the change via ETag and reloads automatically.
2. Via SIGHUP — Send SIGHUP to the server process to reload TLS certificates.

# Manual TLS certificate reload (Unix only)
kill -SIGHUP $(pgrep vouch-server)

Note: SIGHUP only reloads TLS certificates. It does not reload any other configuration fields.

Compliance Mapping

This page maps Vouch features to common compliance frameworks. Vouch’s hardware-backed authentication model satisfies many of the strictest access control requirements across multiple regulatory standards.

NIST 800-53

SOC 2

FedRAMP

HIPAA

AMI STIG Alignment

This page documents how the Vouch Server attestable AMI aligns with the DISA Security Technical Implementation Guide (STIG) for its base operating system, Amazon Linux 2023 (STIG V1R2).

It complements the application-level Compliance Mapping: that page covers the Vouch authentication model, while this page covers the operating system posture of the AMI itself.

Why Vouch does not run the AWS STIG hardening scripts

AWS publishes STIG hardening script bundles (LinuxAWSConfigureSTIG.tgz) that are applied to a running, mutable instance, typically through an SSM document or as EC2 Image Builder STIG-Build-Linux components.

The Vouch AMI does not use them, by design:

• Immutable root filesystem. The root partition is mounted read-only with dm-verity and panic-on-corruption. Runtime remediation scripts either fail to persist (writes land on an overlay tmpfs) or trip the integrity check. All hardening is instead baked in at build time so it lives behind the same dm-verity measurement and TPM attestation (PCR4/7/12) as the rest of the image.
• Minimal attack surface. The scripts can re-enable services and install third-party packages. The AMI deliberately omits SSH, the SSM agent, cloud-init, and ec2-instance-connect; re-introducing them would weaken the posture rather than strengthen it.
• Already met or exceeded. As the mapping below shows, the appliance design satisfies — and in several areas exceeds — the controls those scripts enforce.

Control mapping

Status legend:

• Met — satisfied by an explicit configuration in the image.
• Exceeded — satisfied by a stronger mechanism than the STIG requires.
• N/A by design — the control targets functionality the appliance does not ship (no interactive login, no SSH, no desktop).
• Build-time control — enforced via configuration baked into the image.

Known gap: kernel audit (auditd)

The most material difference from a fully STIG-hardened general-purpose host is the absence of a kernel auditd ruleset. On this appliance the gap is narrow:

• There are no interactive users or shells to audit, which is the primary target of most STIG audit rules.
• Application and credential-issuance events are logged by the Vouch server and forwarded to CloudWatch Logs alongside system journald output.

For deployments with a contractual auditd requirement, auditd and a STIG audit ruleset can be added to the image (package in appliance.kiwi, service enablement in config.sh, rules under etc/audit/rules.d/, and log forwarding in the CloudWatch agent configuration). This is not enabled by default to keep the image minimal and avoid a writable audit-log path on an otherwise userless appliance.

Kernel crypto module validation status

The image runs AL2023 kernel 6.18, where the FIPS 140-3 kernel cryptography is packaged as a standalone modularized kernel module that loads and initializes automatically at boot. Enablement is unchanged by this packaging: fips=1 on the kernel command line plus the FIPS crypto-policy (both baked into the image) remain the mechanism, and the AMI build pipeline boot-verifies fips mode: enabled on the serial console.

Validation status of the layers differs:

• Userspace (server binary): the embedded aws-lc-rs FIPS module is FIPS 140-3 validated.
• Kernel: the AL2023 kernel 6.18 modularized crypto module is under CMVP review (Implementation Under Test), with validation expected to complete in 2027 per AWS. Because the module is decoupled from the kernel image, later kernel updates will no longer invalidate the certificate once it is granted.

For deployments that contractually require a completed CMVP certificate for kernel cryptography today, AL2023 kernel 6.1 carries the currently validated kernel crypto module (certificate #5369) and is supported by AWS for the lifetime of AL2023. Repinning the image is a one-line change to the ami-minimal-kernel6.18 collection in appliance.kiwi, followed by a re-run of the AMI boot-verification workflow.