Vouch Server Operator Guide

This documentation covers deploying, configuring, and operating Vouch Server — the authentication server that issues short-lived credentials after FIDO2 verification with a YubiKey. It covers three deployment patterns: cloud, on-premise, and air-gapped.

For CLI installation, enrollment, integration guides (SSH, AWS, EKS, GitHub, Docker), and OIDC provider documentation (endpoints, tokens, grant types), visit vouch.sh/docs.

What Vouch Server Does

Vouch Server is the backend that makes hardware-backed authentication work:

• OIDC Provider — Issues DPoP-bound access tokens after FIDO2 verification
• SSH Certificate Authority — Signs short-lived Ed25519 certificates
• Credential Broker — Exchanges access tokens for AWS STS credentials
• SCIM Endpoint — Receives user provisioning/de-provisioning from your IdP
• WebAuthn Relying Party — Manages FIDO2 credential registration and assertion

Architecture

Security

Vouch is designed for high-security environments:

• Memory-safe implementation — Written in Rust
• No credential storage — Vouch never sees your private keys
• Cryptographic presence attestation — FIDO2 with user verification
• Short-lived credentials — Minimize blast radius of compromise
• Audit trail — Every credential issuance logged with attestation

Get started with the Deployment Overview.

Deployment Overview

This section covers deploying the Vouch server for your organization. The server is the central authentication backend that handles FIDO2 verification, session management, SSH certificate signing, and OIDC token issuance.

Deployment Checklist

Before deploying, ensure you have:

• Domain name — A domain for your Vouch server (e.g., auth.example.com)
• TLS certificate — Valid certificate for your domain (or use Let’s Encrypt)
• Database — SQLite (single node) or PostgreSQL (multi-node)
• Identity provider — Google Workspace
• JWT secret — Cryptographically random string, minimum 32 characters (or use AWS KMS HMAC)
• SSH CA key (optional) — Ed25519 key pair for signing SSH certificates (or use AWS KMS)
• OIDC signing key (optional) — P-256 EC key for signing ID tokens (or use AWS KMS)

Architecture

                    Internet
                       │
                       ▼
              ┌─────────────────┐
              │  Load Balancer  │
              │  (TLS termination │
              │   or passthrough) │
              └────────┬────────┘
                       │
                       ▼
              ┌─────────────────┐
              │  Vouch Server   │
              │                 │
              │  • Auth Portal  │
              │  • OIDC Provider│
              │  • SSH CA       │
              │  • REST API     │
              └────────┬────────┘
                       │
                       ▼
              ┌─────────────────┐
              │    Database     │
              │                 │
              │  SQLite or      │
              │  PostgreSQL     │
              └─────────────────┘

Deployment Methods

Configuration

All configuration is via environment variables. See the Configuration Reference for the full list.

The minimum configuration requires:

VOUCH_RP_ID=auth.example.com        # Your domain
VOUCH_JWT_SECRET=<64-char-secret>    # Session signing secret
VOUCH_DATABASE_URL=sqlite:vouch.db?mode=rwc  # Database

For production, you’ll also want:

VOUCH_TLS_CERT=<base64-encoded-pem>  # TLS certificate
VOUCH_TLS_KEY=<base64-encoded-pem>   # TLS private key
VOUCH_SSH_CA_KEY=<base64-encoded-pem> # SSH CA key (or VOUCH_SSH_CA_KMS_KEY_ID)
VOUCH_OIDC_ISSUER=https://accounts.google.com  # External IdP
VOUCH_OIDC_CLIENT_ID=...
VOUCH_OIDC_CLIENT_SECRET=...

For AWS deployments, you can use KMS for all signing operations instead of managing local keys. See the Configuration Reference for KMS options.

Sizing

The server is single-process, async (tokio). Per-session memory overhead is minimal (~2 KB for token metadata). The primary bottleneck is database I/O during token issuance and session validation.

Database guidance:

• SQLite — suitable for single-node deployments under ~500 users
• PostgreSQL — recommended for multi-node or >500 users
• Aurora DSQL — for AWS deployments requiring managed infrastructure

Next Steps

1. Database Setup — Choose and configure your database
2. TLS Configuration — Set up HTTPS
3. Configuration Reference — Full environment variable reference
4. Identity Provider Setup — Connect your corporate IdP

Configuration Reference

This chapter describes the Vouch server configuration system, including configuration sources, S3-based configuration, and hot-reload behavior.

Vouch server supports multiple configuration methods with a defined precedence order.

Configuration Sources (Priority Order)

1. S3 Configuration (highest) — JSON file fetched from S3
2. Environment Variables — Standard VOUCH_* prefixed variables
3. Command-line Arguments — Direct CLI arguments

When S3 configuration is enabled, it overrides environment variables. This allows for centralized configuration management with dynamic updates.

S3-Based Configuration

For production deployments, Vouch supports loading configuration from an S3 object. This enables:

• Centralized management — Single source of truth for multi-instance deployments
• Dynamic updates — Configuration changes without server restart (for supported fields)
• TLS hot-reload — Automatic certificate rotation without downtime
• Secrets management — Leverage S3 encryption and IAM for credential protection

Enabling S3 Configuration:

# Required: bucket name
VOUCH_S3_CONFIG_BUCKET=my-bucket

# Optional: object key (default: config/vouch-server.json)
VOUCH_S3_CONFIG_KEY=config/vouch-server.json

# Optional: AWS region (uses default credential chain region if not set)
VOUCH_S3_CONFIG_REGION=us-west-2

# Optional: polling interval in seconds (default: 60)
VOUCH_S3_CONFIG_POLL_INTERVAL=60

S3 Configuration JSON Schema:

{
  "version": 1,
  "listen_addr": "0.0.0.0:443",
  "rp_id": "vouch.example.com",
  "rp_name": "Example Corp",
  "base_url": "https://vouch.example.com",
  "database_url": "postgres://...",
  "jwt_secret": "32+ character secret",
  "tls": {
    "cert": "<base64-encoded PEM certificate>",
    "key": "<base64-encoded PEM private key>"
  },
  "oidc": {
    "issuer_url": "https://accounts.google.com",
    "client_id": "...",
    "client_secret": "..."
  },
  "saml": {
    "idp_metadata_url": "https://idp.example.com/saml/metadata",
    "sp_entity_id": "https://vouch.example.com"
  },
  "allowed_domains": ["example.com"],
  "ssh_ca_key": "<base64-encoded PEM Ed25519 private key>",
  "ssh_ca_kms_key_id": "mrk-...",
  "oidc_signing_key": "<base64-encoded PEM EC P-256 private key>",
  "oidc_signing_kms_key_id": "mrk-...",
  "oidc_rsa_signing_key": "<base64-encoded PEM RSA-3072 private key>",
  "oidc_rsa_signing_kms_key_id": "mrk-...",
  "jwt_hmac_kms_key_id": "mrk-..."
}

See the S3 Configuration Schema for the full field reference.

All certificate and key fields are base64-encoded PEM strings:

# Encode a PEM file for S3 config
base64 -i cert.pem | tr -d '\n'

AWS KMS Signing Keys

As an alternative to managing local key material, Vouch supports AWS KMS for signing operations:

Multi-region keys (mrk- prefix) are recommended for high availability. KMS key IDs can also be set in the S3 config (ssh_ca_kms_key_id, oidc_signing_kms_key_id, jwt_hmac_kms_key_id).

See Key Management for generation and rotation details.

Hot-Reloadable vs Startup-Only Fields

Non-hot-reloadable fields include: jwt_secret, database_url, listen_addr, rp_id, rp_name, session_hours, cors_origins, allowed_domains, dpop.*, OIDC settings, SAML settings, GitHub App settings, SSH CA key, OIDC signing keys, and all KMS key IDs.

Changes to non-hot-reloadable fields in S3 are silently ignored. A server restart is required to apply them.

TLS Certificate Hot-Reload

Vouch supports automatic TLS certificate reloading without dropping connections:

1. Via S3 polling — Update tls.cert and tls.key in S3 config; server detects change via ETag and reloads
2. Via SIGHUP — Send SIGHUP to the server process to reload TLS certificates

# Manual TLS certificate reload (Unix only)
kill -SIGHUP $(pgrep vouch-server)

Note: SIGHUP only reloads TLS certificates. It does not reload any other configuration fields.

Database Setup

Vouch supports three database backends. Choose based on your deployment requirements.

SQLite (Default)

Best for single-node deployments and development. No external dependencies.

VOUCH_DATABASE_URL=sqlite:vouch.db?mode=rwc

The database file is created automatically on first startup. Migrations run automatically.

Recommendations:

• Store the database on a persistent volume
• Set restrictive file permissions: chmod 700 /data
• Back up the file regularly (it’s a single file)

# Create data directory
mkdir -p /data
chmod 700 /data

# Configure
export VOUCH_DATABASE_URL="sqlite:/data/vouch.db?mode=rwc"

PostgreSQL

Best for multi-node deployments, high availability, and production environments.

VOUCH_DATABASE_URL=postgres://user:password@db.example.com:5432/vouch

Setup:

1. Create a PostgreSQL database:

   CREATE DATABASE vouch;
   CREATE USER vouch WITH PASSWORD 'secure-password';
   GRANT ALL PRIVILEGES ON DATABASE vouch TO vouch;
   

2. Configure the connection:

   export VOUCH_DATABASE_URL="postgres://vouch:secure-password@db.example.com:5432/vouch"
   

3. Migrations run automatically on server startup.

Recommendations:

• Use SSL for database connections in production
• Configure connection pooling at the database level
• Set up automated backups

Aurora DSQL

For AWS deployments requiring serverless, distributed SQL with strong consistency.

Aurora DSQL endpoints are auto-detected when the DATABASE_URL hostname contains .dsql. and ends with .on.aws. IAM authentication tokens are generated automatically.

VOUCH_DATABASE_URL=postgres://admin@abcdef123456.dsql.us-east-1.on.aws:5432/vouch

Multi-region configuration uses a dsql_endpoints map in the S3 configuration JSON, resolved via AWS_AZ or AWS_REGION environment variables.

Migrations

Database migrations are embedded in the server binary and run automatically on startup. There is no manual migration step required.

• SQLite migrations: crates/vouch-server/migrations/sqlite/
• PostgreSQL migrations: crates/vouch-server/migrations/postgres/

Backup

Always back up before upgrading the Vouch server, as migrations may modify the schema.

TLS Configuration

Vouch requires HTTPS in production. TLS can be configured directly on the Vouch server or terminated at a load balancer.

Direct TLS (Recommended)

When TLS is configured, the server automatically:

• Listens on port 443 (HTTPS)
• Runs an HTTP redirect server on port 80 (308 redirect to HTTPS)
• Makes the /health endpoint accessible on HTTP (for load balancer health checks)
• Validates the Host header against rp_id to prevent injection attacks
• Ignores VOUCH_LISTEN_ADDR (ports are fixed at 443/80)

Note: Binding to ports 80 and 443 requires CAP_NET_BIND_SERVICE capability on Linux. The RPM/DEB packages configure this automatically.

Configuration

Provide base64-encoded PEM certificates via environment variables:

# Encode your certificate and key
export VOUCH_TLS_CERT="$(base64 -i cert.pem | tr -d '\n')"
export VOUCH_TLS_KEY="$(base64 -i key.pem | tr -d '\n')"

Both VOUCH_TLS_CERT and VOUCH_TLS_KEY must be set together. If only one is provided, the server will fail to start.

TLS Properties

• Protocol: TLS 1.3 only
• Implementation: rustls (no OpenSSL)
• Ciphers: AEAD only (AES-GCM, ChaCha20-Poly1305)

Certificate Hot-Reload

Vouch supports automatic TLS certificate reloading without dropping connections. This is useful for certificate rotation (e.g., Let’s Encrypt renewals).

Via S3 Configuration

If using S3 configuration storage, update the tls.cert and tls.key fields in the S3 config file. The server detects changes via ETag polling and reloads automatically.

Via SIGHUP

Send SIGHUP to the server process to reload TLS certificates:

kill -SIGHUP $(pgrep vouch-server)

Note: SIGHUP only reloads TLS certificates. It does not reload any other configuration.

Self-Signed Certificates (Development)

For development or testing:

# Generate self-signed EC certificate
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
  -keyout tls_key.pem -out tls_cert.pem -days 365 -nodes \
  -subj "/CN=localhost" \
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1"

# Base64 encode for Vouch
export VOUCH_TLS_CERT="$(base64 -i tls_cert.pem | tr -d '\n')"
export VOUCH_TLS_KEY="$(base64 -i tls_key.pem | tr -d '\n')"

Deployment Methods

Choose a deployment method based on your infrastructure:

All methods use the same Vouch server binary and configuration via environment variables. The deployment method only affects how the process is managed and how configuration is provided.

Verification

Regardless of deployment method, verify the server is running:

# Health check
curl -k https://auth.example.com/health
# Expected: {"status":"healthy"}

# SSH CA public key (if configured)
curl -k https://auth.example.com/v1/credentials/ssh/ca
# Expected: ssh-ed25519 AAAA... vouch-ca@...

# OIDC discovery
curl -k https://auth.example.com/.well-known/openid-configuration
# Expected: JSON discovery document

Systemd (Bare Metal)

Deploy Vouch as a systemd service on bare metal servers or VMs.

Install via Package

The RPM and DEB packages include a systemd service unit:

# RPM (RHEL/Fedora/Amazon Linux)
rpm -ivh vouch-server-1.0.0-1.x86_64.rpm

# DEB (Debian/Ubuntu)
dpkg -i vouch-server_1.0.0_amd64.deb

The package installs:

• Binary at /usr/bin/vouch-server
• Systemd unit at /etc/systemd/system/vouch-server.service
• Default config at /etc/vouch/vouch.env
• Data directory at /data (with appropriate permissions)

Configure

Edit the environment file:

sudo cp /etc/vouch/vouch.env /etc/vouch/vouch.env.local
sudo chmod 600 /etc/vouch/vouch.env.local
sudo vi /etc/vouch/vouch.env.local

At minimum, set:

VOUCH_RP_ID=auth.example.com
VOUCH_JWT_SECRET=<your-64-character-secret>
VOUCH_DATABASE_URL=sqlite:/data/vouch.db?mode=rwc
VOUCH_TLS_CERT=<base64-encoded-certificate>
VOUCH_TLS_KEY=<base64-encoded-private-key>

See Configuration Reference for all options.

Start the Service

# Enable and start
sudo systemctl enable --now vouch-server

# Check status
sudo systemctl status vouch-server

# View logs
sudo journalctl -u vouch-server -f

Manual Install (Without Package)

If installing the binary manually:

1. Copy the binary:

   sudo cp vouch-server /usr/bin/
   sudo chmod 755 /usr/bin/vouch-server
   

2. Create a systemd unit:

   # /etc/systemd/system/vouch-server.service
   [Unit]
   Description=Vouch Identity Server
   After=network.target
   
   [Service]
   Type=simple
   User=vouch
   Group=vouch
   EnvironmentFile=/etc/vouch/vouch.env
   ExecStart=/usr/bin/vouch-server
   Restart=on-failure
   RestartSec=5
   
   # Security hardening
   NoNewPrivileges=true
   ProtectSystem=strict
   ProtectHome=true
   ReadWritePaths=/data
   AmbientCapabilities=CAP_NET_BIND_SERVICE
   
   [Install]
   WantedBy=multi-user.target
   

3. Create the service user and directories:

   sudo useradd -r -s /sbin/nologin vouch
   sudo mkdir -p /etc/vouch /data
   sudo chown vouch:vouch /data
   sudo chmod 700 /data
   

4. Reload and start:

   sudo systemctl daemon-reload
   sudo systemctl enable --now vouch-server
   

Upgrading

# Back up database
sudo cp /data/vouch.db /data/vouch.db.backup.$(date +%Y%m%d)

# Upgrade package (migrations run automatically on next startup)
sudo rpm -Uvh vouch-server-1.1.0-1.x86_64.rpm
# or: sudo dpkg -i vouch-server_1.1.0_amd64.deb

# Restart
sudo systemctl restart vouch-server

# Verify
curl -k https://auth.example.com/health

Docker

Deploy Vouch using Docker or Docker Compose.

Docker Run

docker run -d \
  --name vouch-server \
  --restart unless-stopped \
  -p 443:443 \
  -v vouch-data:/data \
  -e VOUCH_RP_ID=auth.example.com \
  -e VOUCH_JWT_SECRET=<your-64-character-secret> \
  -e VOUCH_DATABASE_URL=sqlite:/data/vouch.db?mode=rwc \
  -e VOUCH_TLS_CERT=<base64-encoded-certificate> \
  -e VOUCH_TLS_KEY=<base64-encoded-private-key> \
  ghcr.io/vouch-sh/vouch:latest

Docker Compose

# docker-compose.yml
services:
  vouch-server:
    image: ghcr.io/vouch-sh/vouch:latest
    container_name: vouch-server
    restart: unless-stopped
    ports:
      - "443:443"
      - "80:80"
    volumes:
      - vouch-data:/data
    env_file:
      - vouch.env
    environment:
      VOUCH_DATABASE_URL: sqlite:/data/vouch.db?mode=rwc
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  vouch-data:

Create a vouch.env file:

VOUCH_RP_ID=auth.example.com
VOUCH_JWT_SECRET=<your-64-character-secret>
VOUCH_TLS_CERT=<base64-encoded-certificate>
VOUCH_TLS_KEY=<base64-encoded-private-key>
VOUCH_SSH_CA_KEY=<base64-encoded-ssh-ca-key>

Start:

docker compose up -d
docker compose logs -f vouch-server

With PostgreSQL

# docker-compose.yml
services:
  vouch-server:
    image: ghcr.io/vouch-sh/vouch:latest
    container_name: vouch-server
    restart: unless-stopped
    ports:
      - "443:443"
      - "80:80"
    env_file:
      - vouch.env
    environment:
      VOUCH_DATABASE_URL: postgres://vouch:password@postgres:5432/vouch
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  postgres:
    image: postgres:16
    container_name: vouch-postgres
    restart: unless-stopped
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: vouch
      POSTGRES_USER: vouch
      POSTGRES_PASSWORD: password
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U vouch"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  postgres-data:

Air-Gapped Docker

For air-gapped environments, load the image from a saved archive:

# On connected machine
docker pull ghcr.io/vouch-sh/vouch:1.0.0
docker save ghcr.io/vouch-sh/vouch:1.0.0 -o vouch-server-1.0.0.tar

# Transfer to air-gapped environment

# Load image
docker load < vouch-server-1.0.0.tar

Upgrading

# Pull new image
docker compose pull

# Restart with new image
docker compose up -d

# Verify
docker compose logs -f vouch-server
curl -k https://auth.example.com/health

Kubernetes (Helm)

Deploy Vouch on Kubernetes using the Helm chart.

Prerequisites

• Kubernetes cluster (1.24+)
• Helm 3
• A persistent volume provisioner (for SQLite) or external PostgreSQL

Install

# Install from OCI registry
helm install vouch-server oci://ghcr.io/vouch-sh/charts/vouch-server \
  --version 0.1.0 \
  --namespace vouch \
  --create-namespace \
  --values my-values.yaml

Values

Key values to configure:

# values.yaml
image:
  repository: ghcr.io/vouch-sh/vouch
  pullPolicy: IfNotPresent
  tag: ""  # defaults to chart appVersion

serviceAccount:
  create: true
  annotations: {}

podSecurityContext:
  fsGroup: 65532

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 65532
  seccompProfile:
    type: RuntimeDefault

service:
  type: ClusterIP
  port: 3000

# Environment variables for vouch-server
env:
  VOUCH_LISTEN_ADDR: "0.0.0.0:3000"
  VOUCH_DATABASE_URL: "sqlite:/data/vouch.db?mode=rwc"
  VOUCH_RP_ID: "auth.example.com"
  VOUCH_BASE_URL: "https://auth.example.com"
  RUST_LOG: "info,vouch_server=debug"

# Secret environment variables
# Reference an existing secret containing keys like:
# - VOUCH_JWT_SECRET
# - VOUCH_OIDC_ISSUER
# - VOUCH_OIDC_CLIENT_ID
# - VOUCH_OIDC_CLIENT_SECRET
existingSecret: ""

# Or create a new secret (not recommended for production)
secrets: {}
  # VOUCH_JWT_SECRET: ""

# Ingress
ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    - host: auth.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: vouch-tls
      hosts:
        - auth.example.com

# Resources
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 256Mi

# Persistence (for SQLite)
persistence:
  enabled: true
  existingClaim: ""
  storageClass: ""
  accessMode: ReadWriteOnce
  size: 1Gi
  mountPath: /data

# Health check configuration
healthcheck:
  path: /health
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 3
  failureThreshold: 3

Using Kubernetes Secrets

Create secrets for sensitive values:

kubectl create secret generic vouch-secrets \
  --namespace vouch \
  --from-literal=VOUCH_JWT_SECRET='<your-64-character-secret>' \
  --from-literal=VOUCH_OIDC_ISSUER='https://accounts.google.com' \
  --from-literal=VOUCH_OIDC_CLIENT_ID='...' \
  --from-literal=VOUCH_OIDC_CLIENT_SECRET='...'

Then reference in values:

existingSecret: vouch-secrets

Air-Gapped Kubernetes

For air-gapped environments:

1. Save and transfer the chart:

   helm pull oci://ghcr.io/vouch-sh/charts/vouch-server --version 0.1.0
   # Transfer vouch-server-0.1.0.tgz to air-gapped environment
   

2. Save and transfer the container image:

   docker pull ghcr.io/vouch-sh/vouch:0.1.0
   docker save ghcr.io/vouch-sh/vouch:0.1.0 -o vouch-0.1.0.tar
   # Transfer and load into your private registry
   

3. Install from the local chart:

   helm install vouch-server ./vouch-server-0.1.0.tgz \
     --namespace vouch \
     --create-namespace \
     --set image.repository=registry.internal/vouch \
     --values my-values.yaml
   

Upgrading

helm upgrade vouch-server oci://ghcr.io/vouch-sh/charts/vouch-server \
  --version <new-version> \
  --namespace vouch \
  --values my-values.yaml

Health Checks

The chart configures liveness and readiness probes against the /health endpoint.

Health Checks and Monitoring

Health Endpoint

Vouch exposes a health check endpoint:

GET /health

Response:

{"status": "healthy"}

This endpoint:

• Returns HTTP 200 when the server is operational
• Is accessible over HTTP (port 80) even when TLS is configured, for load balancer health checks
• Does not require authentication

Monitoring Endpoints

Log Format

Vouch uses structured logging via tracing. Set the log level with the RUST_LOG environment variable:

# Production (warnings and errors only)
RUST_LOG=warn

# Standard operation
RUST_LOG=info

# Debugging
RUST_LOG=debug

# Component-specific logging
RUST_LOG=vouch_server=debug,tower_http=info

Audit Events

All authentication and credential issuance events are logged to the database. Key events:

Retention

Configure retention periods for audit events:

# Auth events (login, enrollment) — default 90 days
VOUCH_AUTH_EVENTS_RETENTION_DAYS=730

# OAuth usage events — default 90 days
VOUCH_OAUTH_EVENTS_RETENTION_DAYS=90

Events older than the retention period are cleaned up automatically by the background cleanup task (controlled by VOUCH_CLEANUP_INTERVAL).

Alerting Recommendations

Identity Provider Overview

Vouch uses an external identity provider (IdP) to verify user identity during enrollment. This links a trusted corporate identity to a hardware-bound FIDO2 credential.

Purpose

• Verify the user is a member of your organization during enrollment
• Pull user attributes (email) from your existing identity system
• No separate user database to maintain in Vouch

Supported Protocols

Vouch supports two upstream IdP protocols:

OIDC and SAML are mutually exclusive. Configure one or the other, not both. If both are configured, the server will refuse to start.

OIDC Discovery

When using OIDC, the server automatically discovers authorization, token, and JWKS endpoints by fetching the /.well-known/openid-configuration document from the issuer URL at startup. Any OIDC-compliant provider works — no manual endpoint configuration is needed.

Supported Providers

Configuration

OIDC

VOUCH_OIDC_ISSUER=https://accounts.google.com
VOUCH_OIDC_CLIENT_ID=<your-client-id>
VOUCH_OIDC_CLIENT_SECRET=<your-client-secret>
VOUCH_ALLOWED_DOMAINS=company.com

SAML

VOUCH_SAML_IDP_METADATA_URL=https://idp.example.com/saml/metadata
VOUCH_SAML_SP_ENTITY_ID=https://auth.example.com
VOUCH_SAML_EMAIL_ATTRIBUTE=http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress
VOUCH_ALLOWED_DOMAINS=company.com

Claims and Attribute Mapping

OIDC Claims

SAML Attributes

User Lifecycle

• User exists in external IdP but not Vouch — Enrollment creates Vouch user
• User removed from external IdP — Existing Vouch sessions continue until expiry; re-enrollment blocked

Google Workspace (OIDC)

Configure Google Workspace as your external identity provider for Vouch enrollment via OpenID Connect.

Prerequisites

• Google Workspace admin access
• A verified domain in Google Workspace

Step 1: Create OAuth Client in Google Cloud Console

1. Go to Google Cloud Console
2. Select or create a project
3. Navigate to APIs & Services > Credentials
4. Click Create Credentials > OAuth client ID
5. Select Web application as the application type
6. Configure:
   • Name: Vouch
   • Authorized redirect URIs: https://auth.example.com/oauth/callback
7. Click Create
8. Copy the Client ID and Client Secret

Step 2: Configure Consent Screen

1. Navigate to APIs & Services > OAuth consent screen
2. Select Internal (restricts to your Google Workspace org)
3. Configure:
   • App name: Vouch
   • User support email: your admin email
   • Authorized domains: your Vouch server domain
4. Add scopes: openid, email, profile

Step 3: Configure Vouch

Set the following environment variables on your Vouch server:

VOUCH_OIDC_ISSUER=https://accounts.google.com
VOUCH_OIDC_CLIENT_ID=<your-client-id>.apps.googleusercontent.com
VOUCH_OIDC_CLIENT_SECRET=<your-client-secret>

The server automatically discovers Google’s authorization, token, and JWKS endpoints via OIDC Discovery. No manual endpoint configuration is needed.

Optionally restrict enrollment to specific domains:

VOUCH_ALLOWED_DOMAINS=example.com,subsidiary.com

Step 4: Test

1. Run vouch enroll on a workstation
2. The browser should redirect to Google sign-in
3. After signing in, complete the WebAuthn registration with your YubiKey

Claims Mapping

Troubleshooting

“Access blocked: This app’s request is invalid”

• Verify the redirect URI exactly matches https://<your-vouch-domain>/oauth/callback

“This app is not verified”

• Ensure the OAuth consent screen is set to Internal for Google Workspace

Users from wrong domain can enroll

• Set VOUCH_ALLOWED_DOMAINS to restrict enrollment to specific email domains

Microsoft Entra ID (OIDC)

Configure Microsoft Entra ID (formerly Azure AD) as your upstream identity provider for Vouch enrollment.

Prerequisites

• Microsoft Entra ID tenant with admin access
• An app registration in the Azure portal

Step 1: Register an Application in Entra ID

Follow Microsoft’s app registration guide to create a new app registration:

1. Sign in to the Azure portal
2. Navigate to Microsoft Entra ID > App registrations > New registration
3. Configure:
   • Name: Vouch
   • Supported account types: Accounts in this organizational directory only (single-tenant)
   • Redirect URI: Select Web and enter https://auth.example.com/oauth/callback
4. Click Register

Step 2: Create a Client Secret

1. In the app registration, go to Certificates & secrets > Client secrets
2. Click New client secret
3. Set a description and expiry period
4. Copy the Value (not the Secret ID) immediately — it is only shown once

Step 3: Configure Vouch

Set the following environment variables on your Vouch server:

# Tenant-specific issuer URL (single-tenant)
VOUCH_OIDC_ISSUER=https://login.microsoftonline.com/{tenant-id}/v2.0
VOUCH_OIDC_CLIENT_ID=<application-client-id>
VOUCH_OIDC_CLIENT_SECRET=<client-secret-value>

Replace {tenant-id} with your Entra ID tenant ID (found in Azure portal > Microsoft Entra ID > Overview).

The server automatically discovers authorization, token, and JWKS endpoints from the issuer URL via OIDC Discovery. No manual endpoint configuration is needed.

Optionally restrict enrollment to specific email domains:

VOUCH_ALLOWED_DOMAINS=example.com

Step 4: Test

1. Run vouch enroll on a workstation
2. The browser should redirect to the Microsoft sign-in page
3. After signing in, complete the WebAuthn registration with your YubiKey

Common Pitfalls

Single-tenant vs multi-tenant

Use single-tenant (Accounts in this organizational directory only) to restrict access to your organization. Multi-tenant configurations allow users from any Entra ID tenant to attempt enrollment, which is rarely desired. If you use multi-tenant, ensure VOUCH_ALLOWED_DOMAINS is set to restrict enrollment.

v1 vs v2 endpoints

Always use the v2.0 issuer URL (https://login.microsoftonline.com/{tenant-id}/v2.0). The v1 endpoints use a different token format and are not compatible with standard OIDC discovery.

Redirect URI mismatch

The redirect URI in the app registration must exactly match https://<your-vouch-domain>/oauth/callback. Azure does not support wildcard redirect URIs.

Generic OIDC Provider

Configure any OpenID Connect-compliant identity provider as the upstream IdP for Vouch enrollment.

Prerequisites

• Your IdP must support OIDC Discovery (a /.well-known/openid-configuration endpoint at the issuer URL)
• You need a registered OAuth 2.0 client (client ID and client secret)
• The redirect URI https://<your-vouch-domain>/oauth/callback must be registered with the IdP

Finding the Issuer URL

The issuer URL is the base URL that hosts the OIDC discovery document. You can verify it by fetching {issuer}/.well-known/openid-configuration and confirming it returns a valid JSON document:

curl -s https://your-idp.example.com/.well-known/openid-configuration | jq .issuer

Common issuer URL patterns:

Configuration

Set the following environment variables:

VOUCH_OIDC_ISSUER=https://your-idp.example.com
VOUCH_OIDC_CLIENT_ID=<your-client-id>
VOUCH_OIDC_CLIENT_SECRET=<your-client-secret>

At startup, the server fetches the discovery document from {issuer}/.well-known/openid-configuration and automatically discovers the authorization, token, and JWKS endpoints. No manual endpoint configuration is needed.

Domain Restrictions

Restrict enrollment to specific email domains:

VOUCH_ALLOWED_DOMAINS=example.com,subsidiary.com

If not set, users from any email domain can enroll (provided they authenticate with the upstream IdP).

Tested Providers

The following providers have been tested with Vouch:

Troubleshooting

“Failed to fetch upstream OIDC discovery document”

• Verify the issuer URL is correct and reachable from the server
• Check that the URL uses HTTPS (HTTP is only allowed for localhost)
• Confirm the discovery endpoint returns valid JSON

“Issuer mismatch”

• The issuer field in the discovery document must exactly match the configured VOUCH_OIDC_ISSUER value (trailing slashes matter)

Token errors after authentication

• Ensure the client secret is correct and not expired
• Verify the redirect URI registered with the IdP exactly matches https://<your-vouch-domain>/oauth/callback

SAML 2.0

Configure a SAML 2.0 identity provider for Vouch enrollment. Vouch acts as a SAML Service Provider (SP) with HTTP-POST and HTTP-Redirect bindings.

SAML and OIDC are mutually exclusive. Configure one or the other, not both. If both VOUCH_OIDC_ISSUER and VOUCH_SAML_IDP_METADATA_URL are set, the server will refuse to start.

Environment Variables

SP Metadata

The Vouch server exposes SP metadata for configuring your IdP:

GET https://auth.example.com/saml/metadata

This returns an XML document containing the SP entity ID, Assertion Consumer Service (ACS) URL, and supported bindings. Import this into your IdP or configure the following values manually:

• SP Entity ID: https://auth.example.com (or the value of VOUCH_SAML_SP_ENTITY_ID)
• ACS URL: https://auth.example.com/saml/acs
• Bindings: HTTP-POST (ACS), HTTP-Redirect (AuthnRequest)

Attribute Mapping

Vouch extracts user identity from SAML assertion attributes. By default, it looks for the email address in common attribute names. You can override this with VOUCH_SAML_EMAIL_ATTRIBUTE:

For domain-based enrollment restrictions, set VOUCH_SAML_DOMAIN_ATTRIBUTE to the attribute name containing the user’s domain. If not set, the domain is extracted from the email address.

Configuration Example

VOUCH_SAML_IDP_METADATA_URL=https://login.microsoftonline.com/{tenant-id}/federationmetadata/2007-06/federationmetadata.xml
VOUCH_SAML_SP_ENTITY_ID=https://auth.example.com
VOUCH_SAML_EMAIL_ATTRIBUTE=http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress
VOUCH_ALLOWED_DOMAINS=example.com

Provider-Specific Notes

Microsoft Entra ID

• Metadata URL: https://login.microsoftonline.com/{tenant-id}/federationmetadata/2007-06/federationmetadata.xml
• Email attribute: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress
• Create an Enterprise Application with SAML SSO and set the ACS URL to https://auth.example.com/saml/acs

Okta

• Metadata URL: found in the Okta SAML app’s Sign On tab under Metadata URL
• Email attribute: typically email or configure in the Okta attribute statements
• Set the Single Sign-On URL to https://auth.example.com/saml/acs and Audience URI to your SP entity ID

Google Workspace

• Metadata URL: available from the Google Admin console under Apps > Web and mobile apps > SAML app > Metadata
• Email attribute: email (configure in attribute mapping)
• Add a custom SAML app in the Admin console with ACS URL https://auth.example.com/saml/acs

Troubleshooting

“Failed to fetch SAML IdP metadata”

• Verify the metadata URL is correct and reachable from the server
• Check that the URL returns valid XML (not an HTML login page)
• Ensure the server can make outbound HTTPS requests

Signature verification errors

• Confirm the IdP’s signing certificate in the metadata is current and not expired
• Ensure the server clock is synchronized (NTP). SAML assertions have time-based validity windows.

Email not extracted from assertion

• Check the SAML assertion attributes using debug logging (RUST_LOG=vouch_server=debug)
• Set VOUCH_SAML_EMAIL_ATTRIBUTE to the exact attribute name used by your IdP

“Both OIDC and SAML are configured”

• Only one upstream IdP protocol can be active. Remove either the VOUCH_OIDC_* or VOUCH_SAML_* variables.

Session Management

Vouch sessions are time-limited, DPoP-bound OAuth 2.0 access tokens (ES256 JWTs per RFC 9068) that prove recent hardware presence verification.

Session Lifecycle

1. Creation — vouch login performs FIDO2 assertion with YubiKey touch + PIN
2. Active — Access token stored in agent memory, valid for 8 hours (default)
3. Usage — Credential helpers exchange the access token for service-specific credentials
4. Expiry — Session expires automatically after the configured duration
5. Revocation — vouch logout explicitly ends the session

Session Duration

Default: 8 hours. Configurable via:

VOUCH_SESSION_HOURS=8

Session Storage

Sessions are stored in multiple locations for different access patterns:

Server-Side Session Management

Cleanup

Expired sessions are cleaned up automatically by a background task:

# Cleanup interval in minutes (default: 15, set to 0 to disable)
VOUCH_CLEANUP_INTERVAL=15

Security Properties

• Presence-bound — Every session traces to a FIDO2 assertion with user verification
• Time-limited — Sessions cannot be renewed; a new login is required after expiry
• DPoP-bound — Access tokens are bound to the client’s DPoP key; token theft without the key is useless
• Non-transferable — Sessions are bound to the client that created them
• Audited — Every session creation and usage is logged

Key Management

Vouch uses several cryptographic keys. This page covers their lifecycle and rotation.

Key Inventory

SSH CA Key

The SSH CA key signs all SSH user certificates. Every host that trusts Vouch certificates must have the corresponding public key in TrustedUserCAKeys.

Generation

ssh-keygen -t ed25519 -f ssh_ca_key -N "" -C "vouch-ca@example.com"

Configuration

# Option 1: File path
VOUCH_SSH_CA_KEY_PATH=./ssh_ca_key

# Option 2: Inline (base64-encoded PEM, takes precedence over file)
VOUCH_SSH_CA_KEY="$(base64 -i ssh_ca_key | tr -d '\n')"

# Option 3: AWS KMS (overrides Options 1 and 2)
VOUCH_SSH_CA_KMS_KEY_ID=mrk-1234abcd5678efgh

# Option 4: Disable SSH CA
VOUCH_SSH_CA_KEY_PATH=""

When using KMS, the server calls kms:Sign with Ed25519. The KMS key must be an asymmetric signing key with ECC_EDWARDS_CURVE_25519 key spec. Multi-region keys (mrk- prefix) are recommended for high availability.

Rotation

SSH CA key rotation requires coordinated updates:

1. Generate a new CA key
2. Distribute the new public key to all hosts (add to TrustedUserCAKeys)
3. Update the Vouch server configuration with the new private key
4. Restart the server
5. After all existing certificates expire (max 8 hours), remove the old public key from hosts

Important: During rotation, hosts should trust both old and new CA public keys to avoid disruption.

Public Key Distribution

Retrieve the CA public key:

curl https://auth.example.com/v1/credentials/ssh/ca
# ssh-ed25519 AAAA... vouch-ca@example.com

OIDC Signing Key (ES256)

Used to sign access tokens (RFC 9068) and ID tokens (default algorithm) with ES256.

Configuration

# Option 1: Local key (base64-encoded PEM)
VOUCH_OIDC_SIGNING_KEY="$(base64 -i oidc_signing_key.pem | tr -d '\n')"

# Option 2: AWS KMS (overrides Option 1)
VOUCH_OIDC_SIGNING_KMS_KEY_ID=mrk-abcd1234efgh5678

If neither is set, an ephemeral key is generated on startup. This means tokens cannot be verified after a server restart unless the same key is provided.

When using KMS, the server calls kms:Sign with P-256 ECDSA (ECC_NIST_P256 key spec). Multi-region keys (mrk- prefix) are recommended.

Generation

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:prime256v1 -out oidc_signing_key.pem

Note: You must use openssl genpkey (which produces PKCS#8 format) rather than openssl ecparam -genkey (which produces SEC1 format). The server requires PKCS#8 (-----BEGIN PRIVATE KEY-----).

Rotation

When rotating the OIDC signing key:

1. Generate a new key
2. Update the server configuration
3. Restart the server
4. The JWKS endpoint (/oauth/jwks) automatically serves the new public key
5. Relying parties that cache JWKS will pick up the new key on their next refresh

OIDC RSA Signing Key (RS256)

Used to sign ID tokens with RS256 algorithm per OIDC Core Section 3.1.3.7. RS256 is the default id_token_signed_response_alg in the OIDC specification and must be supported for conformance. Clients can select RS256 via OAuth 2.0 Dynamic Client Registration (id_token_signed_response_alg field).

Access tokens are always signed with ES256 (the OIDC Signing Key above).

Generation

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out oidc_rsa_key.pem

A minimum key size of 3072 bits is enforced. Keys smaller than 3072 bits are rejected at startup.

Configuration

# Option 1: Local key (base64-encoded PEM)
VOUCH_OIDC_RSA_SIGNING_KEY="$(base64 -i oidc_rsa_key.pem | tr -d '\n')"

# Option 2: AWS KMS (overrides Option 1)
VOUCH_OIDC_RSA_SIGNING_KMS_KEY_ID=mrk-rsa1234abcd5678

If neither is set, an ephemeral RSA-3072 key is generated on startup. This means RS256 ID tokens cannot be verified after a server restart unless the same key is provided. A warning is logged when an ephemeral key is generated.

When using KMS, the key must be:

• Key spec: RSA_3072
• Key usage: SIGN_VERIFY
• Signing algorithm: RSASSA_PKCS1_V1_5_SHA_256

Multi-region keys (mrk- prefix) are recommended.

Rotation

When rotating the OIDC RSA signing key:

1. Generate a new RSA-3072 key
2. Update the server configuration
3. Restart the server
4. The JWKS endpoint (/oauth/jwks) automatically serves the new public key
5. Relying parties that cache JWKS will pick up the new key on their next refresh

JWT Secret

Used for signing internal state tokens (authorization codes, WebAuthn challenge state, CSRF tokens) with HS256. Access tokens are signed with the OIDC signing key (ES256) per RFC 9068.

Configuration

# Option 1: Local secret (must be at least 32 characters)
VOUCH_JWT_SECRET="$(openssl rand -base64 48)"

# Option 2: AWS KMS HMAC (eliminates the need for VOUCH_JWT_SECRET)
VOUCH_JWT_HMAC_KMS_KEY_ID=mrk-5678abcd1234efgh

When using KMS, the server uses kms:GenerateMac and kms:VerifyMac with HMAC-SHA256. The KMS key must be a HMAC_256 key type. Multi-region keys (mrk- prefix) are recommended.

Generation (local secret)

openssl rand -base64 48

Rotation

Changing the JWT secret (or KMS key) invalidates all existing sessions. Users must re-authenticate.

1. Generate a new secret or KMS key
2. Update VOUCH_JWT_SECRET or VOUCH_JWT_HMAC_KMS_KEY_ID
3. Restart the server
4. All users must run vouch login again

Document Encryption Key

Used for HPKE (Hybrid Public Key Encryption) of sensitive documents stored alongside the S3 configuration. The private key is encrypted by a KMS key and stored in the S3 config as document_key.

Provisioning

vouch-server generate-document-key --kms-key-id mrk-<your-kms-key-id>

This generates a P-384 EC key pair, encrypts the private key with the specified KMS key, and outputs the document_key JSON block to add to your S3 config.

Configuration

The document_key field in S3 config contains:

{
  "document_key": {
    "kms_key_id": "mrk-<your-kms-key-id>",
    "encrypted_private_key": "<base64-encoded KMS ciphertext>"
  }
}

At startup, the server decrypts the private key via kms:Decrypt and holds the key material in memory for the lifetime of the process.

TLS Certificate

See TLS Configuration for details on TLS certificate management and hot-reload.

SCIM Provisioning

Vouch supports SCIM 2.0 (RFC 7643/7644) for user provisioning and de-provisioning from external identity providers. SCIM is a launch requirement for enterprise deployments.

Setup

The admin API endpoints (/api/v1/org/*) require an authenticated Vouch session from a user with org admin privileges. The server accepts the access token via Authorization: Bearer <token>, Authorization: DPoP <token>, or the vouch_session cookie.

Prerequisites:

• Your user account must have the is_org_admin flag set in the database
• You must be assigned to an organization (org_id)

1. Create a SCIM token

Log in with vouch login, then create a SCIM token using the cookie file or token command:

# Using cookie file (written automatically on login)
curl -X POST https://auth.example.com/api/v1/org/scim-tokens \
  -b ~/.vouch/cookie.txt \
  -H "Content-Type: application/json" \
  -d '{"description": "SCIM integration", "expires_in_days": 90}'

# Or using the token command
curl -X POST https://auth.example.com/api/v1/org/scim-tokens \
  -H "Authorization: Bearer $(vouch credential token)" \
  -H "Content-Type: application/json" \
  -d '{"description": "SCIM integration", "expires_in_days": 90}'

The response includes a token prefixed vouch_scim_.... This token is shown once and cannot be retrieved again.

2. Configure your IdP

Enter the following in your IdP’s SCIM configuration:

• SCIM endpoint URL: https://auth.example.com/scim/v2/
• Bearer token: the vouch_scim_... token from step 1

3. Manage tokens

# List active SCIM tokens
curl -b ~/.vouch/cookie.txt \
  https://auth.example.com/api/v1/org/scim-tokens

# Revoke a SCIM token
curl -X DELETE -b ~/.vouch/cookie.txt \
  https://auth.example.com/api/v1/org/scim-tokens/<token-id>

De-Provisioning Behavior

When a user is de-provisioned via SCIM (e.g., employee leaves the organization):

Key principle: De-provisioning is immediate and complete. When someone leaves via SCIM, they lose all Vouch access instantly — no waiting for session expiration.

#![allow(unused)]
fn main() {
// SCIM de-provision handling (DELETE /scim/v2/Users/:id)
async fn delete_user(user_id: &str) -> Result<()> {
    // 1. Invalidate all active sessions immediately
    db::delete_sessions_for_user(&db, user_id).await?;

    // 2. Revoke all SSH certificates for this user
    db::revoke_all_ssh_certificates_for_user(&db, user_id, Some("User deleted via SCIM"), Some("scim")).await?;

    // 3. Delete user (cascades to authenticators)
    db::delete_user(&db, user_id).await?;

    // 4. Log audit event
    db::insert_scim_audit(&db, "delete", "User", user_id, Some(&token_id), Some(&details)).await?;

    Ok(())
}
}

SCIM Endpoint Authentication

SCIM endpoints require bearer token authentication:

Endpoint: POST /scim/v2/Users, DELETE /scim/v2/Users/:id, etc.

Authentication:

• Bearer token in Authorization header
• Token generated via the admin API (POST /api/v1/org/scim-tokens)
• Tokens are long-lived but can be rotated/revoked
• Separate token per IdP integration

# Example SCIM request
curl -X DELETE https://vouch.example.com/scim/v2/Users/usr_abc123 \
  -H "Authorization: Bearer scim_token_xyz789" \
  -H "Content-Type: application/scim+json"

Token Security:

• Tokens are hashed (SHA-256) before storage
• Shown once at creation, never retrievable after
• Bound to specific organization
• Minimum 256 bits of entropy

SCIM Audit Logging

All SCIM operations are logged for compliance and security monitoring:

SCIM vs Manual Enrollment

Note: SCIM pre-provisioning creates a user record, but they still cannot authenticate until they physically enroll a hardware FIDO2 authenticator. The security model remains: no credential without hardware.

S3 Configuration Storage

When using S3-based configuration, additional security considerations apply. This chapter covers bucket requirements, protected fields, polling behavior, and key encoding.

S3 Bucket Requirements

Recommended S3 Bucket Policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::ACCOUNT:role/vouch-server"},
      "Action": ["s3:GetObject", "s3:HeadObject"],
      "Resource": "arn:aws:s3:::my-bucket/config/vouch-server.json"
    }
  ]
}

Protected Configuration Fields

Only TLS certificates can be hot-reloaded at runtime via S3 polling. All other configuration fields are silently ignored during runtime S3 polling and require a server restart to take effect.

S3 Polling Security

• ETag-based polling — HEAD request checks for changes before full GET
• Fail-open on polling errors — If S3 becomes unreachable during operation, server continues with current config
• Fail-fast on startup — If S3 is unreachable at startup when configured, server fails to start
• No caching of stale config — Config is only updated when successfully fetched and parsed

Base64 Encoding for Keys

All private keys and certificates in S3 config must be base64-encoded:

• Standard base64 or URL-safe base64 accepted
• Prevents JSON parsing issues with PEM format
• Decoded and validated before use

Backup and Recovery

What to Back Up

Backup Strategy

Database

SQLite:

# Simple file copy (stop writes first or use backup API)
cp /data/vouch.db /backup/vouch.db.$(date +%Y%m%d_%H%M%S)

# Or use SQLite backup command (safe during writes)
sqlite3 /data/vouch.db ".backup '/backup/vouch.db.backup'"

PostgreSQL:

pg_dump -Fc vouch > /backup/vouch.$(date +%Y%m%d_%H%M%S).dump

Frequency: Daily minimum. More frequent for high-activity deployments.

Cryptographic Keys

Back up all keys to a secure, offline location:

# SSH CA key
cp ssh_ca_key /secure-backup/ssh_ca_key

# OIDC signing key (ES256)
cp oidc_signing_key.pem /secure-backup/oidc_signing_key.pem

# OIDC RSA signing key (RS256) — if configured
cp oidc_rsa_key.pem /secure-backup/oidc_rsa_key.pem

Store key backups:

• Encrypted at rest
• In a separate location from the server
• With restricted access (minimum two-person rule for production)

Recovery Procedures

Full Server Recovery

1. Deploy new server with the same configuration
2. Restore database from backup
3. Restore cryptographic keys (SSH CA, OIDC signing, JWT secret)
4. Start the server — migrations run automatically if needed
5. Verify: curl https://auth.example.com/health

Lost SSH CA Key

If the SSH CA key is lost and no backup exists:

1. Generate a new SSH CA key
2. Distribute the new public key to all SSH hosts
3. Configure Vouch with the new key
4. All users must run vouch login to get new certificates

Lost JWT Secret

If the JWT secret changes (lost or compromised):

1. Set the new VOUCH_JWT_SECRET
2. Restart the server
3. All existing sessions are invalidated
4. Users must run vouch login again

Database Corruption

1. Stop the server
2. Restore from backup
3. Users who enrolled after the backup will need to re-enroll
4. Start the server

Disaster Recovery Testing

Periodically test your recovery procedures:

1. Restore a database backup to a test environment
2. Start a test server with production keys
3. Verify enrollment, login, and credential flows
4. Document any issues and update procedures

Software Updates

Update Procedure

Pre-Update

1. Read the release notes for breaking changes
2. Back up the database before upgrading
3. Test in staging if possible

Server Update

# Back up database
cp /data/vouch.db /data/vouch.db.pre-upgrade

# Update via package manager
sudo apt upgrade vouch-server    # Debian/Ubuntu
sudo dnf upgrade vouch-server    # RHEL/Fedora

# Or via Docker
docker pull ghcr.io/vouch-sh/vouch:latest
docker compose up -d

# Or via Helm
helm upgrade vouch-server oci://ghcr.io/vouch-sh/charts/vouch-server \
  --version <new-version> --namespace vouch

# Verify
curl -k https://auth.example.com/health

Database migrations run automatically on startup. No manual migration steps are needed.

Client Update

# macOS
brew upgrade vouch

# Linux
sudo apt upgrade vouch    # Debian/Ubuntu
sudo dnf upgrade vouch    # RHEL/Fedora

Rollback

If an update causes issues:

1. Stop the server
2. Restore the database from pre-upgrade backup
3. Install the previous version
4. Start the server

Note: Database migrations may not be reversible. Always back up before upgrading.

Version Compatibility

• The server is backward-compatible with older CLI versions
• Upgrade the server first, then clients
• Major version bumps may require simultaneous client updates (documented in release notes)

Release Channels

Troubleshooting

Common Issues

Server Connection Issues

“Connection refused” or timeouts

1. Check server health: curl -k https://auth.example.com/health
2. Check DNS resolution: dig auth.example.com
3. Check TLS: openssl s_client -connect auth.example.com:443
4. Check firewall rules (port 443 must be accessible)

SCIM Provisioning Issues

User not de-provisioned

• Verify the SCIM bearer token is valid and not expired
• Check the SCIM audit log for errors
• Confirm the IdP is sending DELETE requests to the correct endpoint

SCIM token rejected

• Tokens are shown once at creation and cannot be retrieved after
• Generate a new token via the admin API (POST /api/v1/org/scim-tokens) and update the IdP configuration

Identity Provider Issues

“Failed to fetch upstream OIDC discovery document”

1. Verify VOUCH_OIDC_ISSUER is correct and reachable from the server: curl -s $VOUCH_OIDC_ISSUER/.well-known/openid-configuration | jq .issuer
2. Check that the issuer URL uses HTTPS (HTTP is only allowed for localhost)
3. Ensure the server can make outbound HTTPS requests (check firewall/proxy)

“Issuer mismatch” during OIDC discovery

• The issuer field in the discovery document must exactly match VOUCH_OIDC_ISSUER (trailing slashes matter)
• Some providers require a trailing slash (e.g., Auth0: https://tenant.auth0.com/)

“Failed to fetch SAML IdP metadata”

• Verify VOUCH_SAML_IDP_METADATA_URL is correct and reachable
• Check that the URL returns XML, not an HTML login page
• Ensure the server can make outbound HTTPS requests

SAML signature verification errors

• Confirm the IdP’s signing certificate in the metadata is current and not expired
• Ensure the server clock is synchronized via NTP — SAML assertions have time-based validity windows (typically 5 minutes of skew tolerance)
• Check the IdP assertion signing algorithm matches what the server expects

“Both OIDC and SAML are configured”

• OIDC and SAML are mutually exclusive — remove either the VOUCH_OIDC_* or VOUCH_SAML_* variables
• In S3 config, remove either the oidc or saml block

Debug Logging

Enable verbose logging for troubleshooting:

# Server
RUST_LOG=debug vouch-server

For component-specific logging:

RUST_LOG=vouch_server=debug

Getting Help

• GitHub Issues — Bug reports
• GitHub Discussions — Questions
• Security Issues — Security vulnerabilities

Incident Response

This chapter describes Vouch’s incident severity classification, response procedures, and communication channels for security events.

Severity Levels

Response Procedure

1. Triage — Assess severity and scope
2. Contain — Revoke affected credentials, disable vulnerable features
3. Investigate — Root cause analysis
4. Remediate — Deploy fix
5. Communicate — Notify affected users
6. Review — Post-incident analysis

Communication Channels

• Security advisories: https://vouch.sh/security
• CVE assignments: Via GitHub Security Advisories
• Status page: https://status.vouch.sh

Air-Gapped Deployment

This chapter covers deploying Vouch in environments with no internet connectivity, such as defense contractors, government agencies, financial services, and critical infrastructure.

Status: Planned – This document describes the air-gapped deployment architecture for Vouch. Server and CLI packages are available from packages.vouch.sh, and the core components (SSH CA, FIDO2 authentication) exist today. However, air-gap-specific CLI commands (e.g., vouch enroll --airgap) and automation scripts are not yet implemented.

Overview

In an air-gapped environment:

• No SaaS services available
• Updates delivered via sneakernet
• Internal identity provider (no Google Workspace)
• Time sync from isolated NTP or GPS

Vouch’s built-in SSH CA and local-first architecture make it well-suited for these constraints.

Architecture

+--------------------------------------------------------------------------+
|                          AIR-GAPPED ENCLAVE                              |
|                                                                          |
|  +--------------------------------------------------------------------+  |
|  |                     On-Premises Vouch Stack                        |  |
|  |                                                                    |  |
|  |  +--------------+  +----------------+  +-----------------------+   |  |
|  |  |   Vouch      |  |   Built-in     |  |       SQLite          |   |  |
|  |  |   Server     |  |   SSH CA       |  |                       |   |  |
|  |  |              |  |                |  |  * Users & credentials |   |  |
|  |  |  * WebAuthn  |  |  * Ed25519 CA  |  |  * Sessions           |   |  |
|  |  |  * OIDC      |  |  * SSH certs   |  |  * Audit logs         |   |  |
|  |  |  * Sessions  |  |  * 8hr TTL     |  |                       |   |  |
|  |  +--------------+  +----------------+  +-----------------------+   |  |
|  |         |                  |                      |                |  |
|  |         +------------------+----------------------+                |  |
|  |                            |                                       |  |
|  +----------------------------+---------------------------------------+  |
|                               |                                          |
|                               | Internal Network Only                    |
|                               v                                          |
|  +--------------------------------------------------------------------+  |
|  |                        Workstations                                |  |
|  |                                                                    |  |
|  |  +--------------+  +--------------+  +-------------------------+   |  |
|  |  | Workstation  |  | Workstation  |  |   Protected Resources   |   |  |
|  |  |              |  |              |  |                         |   |  |
|  |  | * vouch CLI  |  | * vouch CLI  |  |  * SSH servers          |   |  |
|  |  | * YubiKey    |  | * YubiKey    |  |  * Internal apps        |   |  |
|  |  | * Certs      |  | * Certs      |  |  * Databases            |   |  |
|  |  +--------------+  +--------------+  +-------------------------+   |  |
|  |                                                                    |  |
|  +--------------------------------------------------------------------+  |
|                                                                          |
|  +--------------------------------------------------------------------+  |
|  |                       Time Infrastructure                          |  |
|  |  +------------+     +-----------------+                            |  |
|  |  | GPS Time   |---->|  Internal NTP   |----> All hosts             |  |
|  |  | Receiver   |     |  (stratum 1)    |                            |  |
|  |  +------------+     +-----------------+                            |  |
|  +--------------------------------------------------------------------+  |
+--------------------------------------------------------------------------+
                                    |
                                    | Air Gap (sneakernet)
                                    v
+--------------------------------------------------------------------------+
|                         CONNECTED ENVIRONMENT                            |
|                                                                          |
|  * Signed software packages (from packages.vouch.sh)                     |
|  * CA certificate updates                                                |
|  * (Optional) Audit log export                                           |
+--------------------------------------------------------------------------+

Identity Provider Considerations

In an air-gapped environment, you cannot use external identity providers like Google Workspace for enrollment. There are several options for handling user identity:

• Self-hosted OIDC provider — Deploy an internal OIDC-compliant IdP inside the enclave (e.g., Keycloak, Dex, or Microsoft AD FS). Configure Vouch Server’s VOUCH_OIDC_ISSUER, VOUCH_OIDC_CLIENT_ID, and VOUCH_OIDC_CLIENT_SECRET environment variables to point to the internal IdP.
• No external IdP — The VOUCH_OIDC_* environment variables are optional, so Vouch Server can start without an upstream OIDC provider. See the Environment Variables reference for details.

Needs product decision: The exact enrollment workflow without an external IdP (e.g., admin-initiated enrollment, local credential bootstrapping) is not yet defined. This section will be updated once the air-gapped enrollment flow is finalized.

Prerequisites

Hardware

• Servers for Vouch stack (VMs or bare metal)
• YubiKey 5 series for each user (firmware 5.2+)
• GPS receiver for time sync (recommended)
• USB drives for sneakernet transfers

Software (Pre-downloaded)

• Vouch Server packages (RPM/DEB from packages.vouch.sh)
• vouch CLI packages (RPM/DEB from packages.vouch.sh)
• Container images and/or Helm charts (for Kubernetes deployments)

Installation

This chapter walks through the complete installation procedure for deploying Vouch in an air-gapped environment, from downloading packages on a connected machine through enrolling users on the isolated network.

Step 1: Download Packages for Offline Transfer

On a connected machine, download the required packages from packages.vouch.sh:

# Import Vouch GPG signing key
curl -fsSL https://packages.vouch.sh/gpg/vouch.asc | gpg --import

# Download server RPM
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-server-1.0.0-1.x86_64.rpm

# Download CLI RPM (for each workstation architecture)
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-1.0.0-1.x86_64.rpm
curl -LO https://packages.vouch.sh/rpm/aarch64/vouch-1.0.0-1.aarch64.rpm

# For Debian/Ubuntu workstations
curl -LO https://packages.vouch.sh/apt/vouch_1.0.0_amd64.deb
curl -LO https://packages.vouch.sh/apt/vouch_1.0.0_arm64.deb

For container-based or Kubernetes deployments, also download:

# Pull and save container image
docker pull ghcr.io/vouch-sh/vouch:1.0.0
docker save ghcr.io/vouch-sh/vouch:1.0.0 -o vouch-server-1.0.0.tar

# Download Helm chart (for Kubernetes)
helm pull oci://ghcr.io/vouch-sh/charts/vouch-server --version 0.1.0

Generate checksums for verification after transfer:

sha256sum vouch-server-*.rpm vouch-*.rpm vouch-*.deb vouch-server-*.tar > SHA256SUMS
gpg --detach-sign SHA256SUMS

Transfer all files to the air-gapped environment via approved media.

Step 2: Verify Package Integrity

On the air-gapped network:

# Import Vouch GPG signing key (transferred separately, verified out-of-band)
gpg --import vouch-release-key.pub

# Verify checksums
gpg --verify SHA256SUMS.sig SHA256SUMS
sha256sum -c SHA256SUMS

# Verify RPM signatures
rpm -K vouch-server-1.0.0-1.x86_64.rpm
rpm -K vouch-1.0.0-1.x86_64.rpm

Step 3: Install Packages

RPM-based installation (recommended for bare metal/VM):

# Install server
rpm -ivh vouch-server-1.0.0-1.x86_64.rpm

# Install CLI on workstations
rpm -ivh vouch-1.0.0-1.x86_64.rpm

DEB-based installation:

# Install CLI on Debian/Ubuntu workstations
dpkg -i vouch_1.0.0_amd64.deb

Container-based installation:

# Load container image into local Docker registry
docker load < vouch-server-1.0.0.tar

# Verify image loaded
docker images | grep vouch

Step 4: Secure Key Generation

This is a critical security operation. Follow your organization’s key ceremony procedures.

Vouch requires several cryptographic keys for different purposes. All key generation should be performed on a trusted, air-gapped workstation. See the Key Ceremony chapter for detailed instructions on generating each key.

Key Overview

*Auto-generates ephemeral key if not provided (not recommended for production).

Note: All PEM-formatted keys and certificates must be base64-encoded when passed via environment variables. This ensures proper handling of newlines and special characters.

Step 5: Database Setup

Vouch uses SQLite by default, which is suitable for single-node deployments. The database is created automatically on first startup.

# SQLite (default, single-node)
export VOUCH_DATABASE_URL="sqlite:/data/vouch.db?mode=rwc"

# Create data directory with appropriate permissions
mkdir -p /data
chmod 700 /data

For high-availability deployments, a local PostgreSQL instance is supported:

# PostgreSQL (multi-node, must be reachable on the internal network)
export VOUCH_DATABASE_URL="postgres://user:password@db.internal:5432/vouch"

Database migrations run automatically on server startup.

Step 6: Configure Vouch Server

Vouch is configured entirely through environment variables. Create a secure environment file:

# Create environment file (chmod 600 after editing)
cat > /etc/vouch/vouch.env << 'EOF'
# =============================================================================
# Vouch Server Configuration - Air-Gapped Environment
# =============================================================================

# -----------------------------------------------------------------------------
# Required Configuration
# -----------------------------------------------------------------------------

# JWT signing secret (minimum 32 characters)
VOUCH_JWT_SECRET=<your-64-character-secret-here>

# Relying Party configuration
VOUCH_RP_ID=auth.internal
VOUCH_RP_NAME=Vouch (Air-Gapped)

# Database
VOUCH_DATABASE_URL=sqlite:/data/vouch.db?mode=rwc

# -----------------------------------------------------------------------------
# Network Configuration
#
# Development (no TLS):
#   Server listens on VOUCH_LISTEN_ADDR (default: 0.0.0.0:3000)
#
# Production (TLS enabled):
#   Server automatically listens on port 443 (HTTPS) and port 80 (HTTP redirect)
#   VOUCH_LISTEN_ADDR is ignored when TLS is configured
#   HTTP requests on port 80 are 308 redirected to HTTPS on port 443
#   The /health endpoint is accessible on HTTP (for load balancer health checks)
#   Host header is validated against rp_id to prevent injection attacks
#   Requires CAP_NET_BIND_SERVICE capability (handled by packaging scripts)
# -----------------------------------------------------------------------------

# Listen address (used only when TLS is NOT configured)
VOUCH_LISTEN_ADDR=0.0.0.0:3000

# Base URL (how clients reach the server)
VOUCH_BASE_URL=https://auth.internal

# -----------------------------------------------------------------------------
# TLS Configuration (base64-encoded PEM)
# Generate with: base64 -i cert.pem | tr -d '\n'
# -----------------------------------------------------------------------------

VOUCH_TLS_CERT=<base64-encoded-certificate>
VOUCH_TLS_KEY=<base64-encoded-private-key>

# -----------------------------------------------------------------------------
# SSH CA Configuration (base64-encoded PEM)
# Generate with: base64 -i ssh_ca_key | tr -d '\n'
# -----------------------------------------------------------------------------

# SSH CA private key (base64-encoded PEM, takes precedence over path)
VOUCH_SSH_CA_KEY=<base64-encoded-ssh-ca-private-key>

# Or use a file path instead (file contains raw PEM, not base64):
# VOUCH_SSH_CA_KEY_PATH=/secrets/ssh_ca_key

# -----------------------------------------------------------------------------
# OIDC Provider Configuration
# Generate with: base64 -i oidc_signing_key.pem | tr -d '\n'
# -----------------------------------------------------------------------------

# Vouch acts as an OIDC provider - this key signs the ID tokens (base64-encoded PEM)
VOUCH_OIDC_SIGNING_KEY=<base64-encoded-oidc-signing-key>

# -----------------------------------------------------------------------------
# External Identity Provider (Optional)
# For enrollment via external IdP (e.g., on-premises OIDC provider)
# -----------------------------------------------------------------------------

# VOUCH_OIDC_ISSUER=https://idp.internal
# VOUCH_OIDC_CLIENT_ID=vouch-client
# VOUCH_OIDC_CLIENT_SECRET=<client-secret>

# -----------------------------------------------------------------------------
# Session Configuration
# -----------------------------------------------------------------------------

# Session duration (default: 8 hours)
VOUCH_SESSION_HOURS=8

# Device code settings (for CLI enrollment)
VOUCH_DEVICE_CODE_EXPIRES=600
VOUCH_DEVICE_POLL_INTERVAL=5

# -----------------------------------------------------------------------------
# Security Configuration
# -----------------------------------------------------------------------------

# Allowed email domains for enrollment (comma-separated)
VOUCH_ALLOWED_DOMAINS=internal,company.local

# DPoP (Demonstrating Proof of Possession)
VOUCH_DPOP_ENABLED=true
VOUCH_DPOP_NONCE_REQUIRED=false
VOUCH_DPOP_MAX_AGE=300

# -----------------------------------------------------------------------------
# Audit and Retention
# -----------------------------------------------------------------------------

# Cleanup interval (minutes, 0 to disable)
VOUCH_CLEANUP_INTERVAL=15

# Event retention (days)
VOUCH_AUTH_EVENTS_RETENTION_DAYS=730
VOUCH_OAUTH_EVENTS_RETENTION_DAYS=90

# -----------------------------------------------------------------------------
# Branding (Optional)
# -----------------------------------------------------------------------------

VOUCH_ORG_NAME=Your Organization
EOF

# Secure the environment file
chmod 600 /etc/vouch/vouch.env

Environment Variables Reference

Step 7: Deploy Services

There are several options for deploying the Vouch server.

Option A: Systemd Service (RPM Install)

If you installed via RPM, the vouch-server systemd service is configured automatically:

# Configure environment
cp /etc/vouch/vouch.env /etc/vouch/vouch.env.local
# Edit /etc/vouch/vouch.env.local with your settings

# Start and enable the service
systemctl enable --now vouch-server

# Check status
systemctl status vouch-server

# View logs
journalctl -u vouch-server -f

Option B: Docker Compose

# docker-compose.yml
services:
  vouch-server:
    image: ghcr.io/vouch-sh/vouch:1.0.0
    container_name: vouch-server
    restart: unless-stopped
    ports:
      - "443:443"
    volumes:
      - vouch-data:/data
      - /etc/vouch/secrets:/secrets:ro
    env_file:
      - /etc/vouch/vouch.env
    environment:
      # Override or add environment variables here
      VOUCH_DATABASE_URL: sqlite:/data/vouch.db?mode=rwc
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  vouch-data:

# Start services
docker-compose up -d

# Verify container is running
docker-compose ps

# Check logs for startup errors
docker-compose logs -f vouch-server

Option C: Helm Chart (Kubernetes)

A Helm chart is available for Kubernetes deployments. After transferring the chart archive to the air-gapped environment:

# Install from the downloaded chart archive
helm install vouch-server vouch-server-0.1.0.tgz \
  --namespace vouch \
  --create-namespace \
  --set image.repository=vouch-server \
  --set image.tag=1.0.0 \
  --values my-values.yaml

See the chart’s values.yaml for all configurable options including secrets, ingress, and persistent storage.

Verify Deployment

Regardless of deployment method:

# Verify health endpoint
curl -k https://auth.internal/health
# Expected: {"status":"healthy"}

# Verify SSH CA is loaded (if configured)
curl -k https://auth.internal/v1/credentials/ssh/ca
# Expected: ssh-ed25519 AAAA... vouch-ca@auth.internal

Step 8: Distribute CA Public Key

The SSH CA public key must be trusted by all SSH servers in the air-gapped environment:

# Fetch CA public key via API
curl -k https://auth.internal/v1/credentials/ssh/ca > vouch-ca.pub

# Copy to all SSH servers
scp vouch-ca.pub root@server:/etc/ssh/vouch-ca.pub

# Configure SSH server to trust the CA
echo "TrustedUserCAKeys /etc/ssh/vouch-ca.pub" >> /etc/ssh/sshd_config

# Optionally, configure authorized principals
echo "AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u" >> /etc/ssh/sshd_config

# Restart SSH daemon
systemctl restart sshd

AuthorizedPrincipals Setup (Optional but Recommended):

# Create principals directory
mkdir -p /etc/ssh/auth_principals

# For each user, create a file with allowed principals
# Vouch issues certificates with two principals: email and username
echo "john@company.internal" > /etc/ssh/auth_principals/john
echo "john" >> /etc/ssh/auth_principals/john

Step 9: Configure CLI for Air-Gap

# ~/.vouch/config.json
{
  "server_url": "https://auth.internal",
  "ca_cert_path": "/etc/vouch/root-ca.crt"
}

Or via environment:

export VOUCH_SERVER=https://auth.internal
export VOUCH_CA_CERT=/etc/vouch/root-ca.crt

Step 10: Enroll Users

Important: Enrollment requires browser access to the Vouch server’s web UI on the internal network. CLI-only enrollment (vouch enroll --airgap) is not yet implemented.

Each user:

1. Opens a browser to https://auth.internal/enroll
2. Authenticates via the configured identity provider (or direct registration if no external IdP is configured)
3. Registers their YubiKey through the browser’s WebAuthn prompt (touch + PIN)

After enrollment, daily login uses the CLI (vouch login) with no browser required.

Key Ceremony

This chapter details the cryptographic key generation procedures required for an air-gapped Vouch deployment. All key generation should be performed on a trusted, air-gapped workstation following your organization’s key ceremony procedures.

JWT Secret Generation (Required)

The JWT secret is used to sign all OAuth tokens and session cookies. It must be at least 32 characters.

# Generate cryptographically secure 64-character secret
openssl rand -base64 48

# Alternative using /dev/urandom
head -c 48 /dev/urandom | base64

# Store securely - this will be VOUCH_JWT_SECRET

Security Notes:

• Use a minimum of 32 characters (48+ recommended)
• Never reuse secrets across environments
• Rotate periodically (requires re-authentication of all users)

SSH CA Key Generation (Ed25519)

The SSH CA signs user SSH certificates. If not provided, SSH certificate issuance will be disabled.

# Generate Ed25519 SSH CA key pair (no passphrase for automated use)
ssh-keygen -t ed25519 -f ssh_ca_key -N "" -C "vouch-ca@auth.internal"

# Set restrictive permissions
chmod 600 ssh_ca_key

# Verify key type and fingerprint
ssh-keygen -l -f ssh_ca_key
# Output: 256 SHA256:xxxx vouch-ca@auth.internal (ED25519)

# View public key (for distribution to SSH servers)
cat ssh_ca_key.pub

Environment variable format (base64-encoded):

# Option 1: Provide base64-encoded key content (preferred for containers)
export VOUCH_SSH_CA_KEY="$(base64 -i ssh_ca_key | tr -d '\n')"

# Option 2: Provide path to key file (file contains raw PEM, not base64)
export VOUCH_SSH_CA_KEY_PATH="/secrets/ssh_ca_key"

Key Storage Options:

• HSM (recommended for high-security) – Store in hardware security module
• Encrypted file with split knowledge – Two administrators hold partial keys
• YubiKey PIV (for smaller deployments) – Store on hardware token

OIDC Signing Key Generation (P-256 ECDSA)

The OIDC signing key signs ID tokens using ES256 algorithm. If not provided, an ephemeral key is generated on each server restart (not recommended for production as it invalidates all existing tokens).

# Generate P-256 EC private key in PKCS#8 format
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:prime256v1 -out oidc_signing_key.pem

# Set restrictive permissions
chmod 600 oidc_signing_key.pem

# Verify key type
openssl ec -in oidc_signing_key.pem -text -noout 2>/dev/null | head -3
# Output should include: Private-Key: (256 bit, prime256v1)

# Extract public key (for debugging/verification)
openssl ec -in oidc_signing_key.pem -pubout -out oidc_signing_key.pub

Environment variable format (base64-encoded):

# Provide base64-encoded PEM content
export VOUCH_OIDC_SIGNING_KEY="$(base64 -i oidc_signing_key.pem | tr -d '\n')"

OIDC RSA Signing Key Generation (RSA-3072)

The OIDC RSA signing key signs ID tokens with RS256 algorithm per OIDC Core Section 3.1.3.7. This is optional but recommended for OIDC specification conformance. If not provided, an ephemeral key is generated on each server restart.

# Generate RSA-3072 private key in PKCS#8 format
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out oidc_rsa_key.pem

# Set restrictive permissions
chmod 600 oidc_rsa_key.pem

# Verify key type and size
openssl rsa -in oidc_rsa_key.pem -text -noout 2>/dev/null | head -3
# Output should include: Private-Key: (3072 bit)

Environment variable format (base64-encoded):

# Provide base64-encoded PEM content
export VOUCH_OIDC_RSA_SIGNING_KEY="$(base64 -i oidc_rsa_key.pem | tr -d '\n')"

TLS Certificate Generation

For production, use certificates signed by your internal CA. For testing, self-signed certificates can be used.

# Generate EC private key and self-signed certificate
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 \
  -keyout tls_key.pem -out tls_cert.pem -days 365 -nodes \
  -subj "/CN=auth.internal" \
  -addext "subjectAltName=DNS:auth.internal,DNS:localhost"

# Set restrictive permissions
chmod 600 tls_key.pem

# Verify certificate
openssl x509 -in tls_cert.pem -text -noout | head -15

Environment variable format (base64-encoded):

# Base64 encode for environment variables
export VOUCH_TLS_CERT="$(base64 -i tls_cert.pem | tr -d '\n')"
export VOUCH_TLS_KEY="$(base64 -i tls_key.pem | tr -d '\n')"

Key Security Best Practices

1. File Permissions: Always use chmod 600 for private keys
2. Never Commit Keys: Add *.pem, *_key, *.key to .gitignore
3. Audit Key Access: Log all access to key material
4. Backup Securely: Store encrypted backups in separate secure location
5. Document Fingerprints: Record key fingerprints in secure documentation
6. Key Rotation: Plan for periodic rotation (SSH CA annually, JWT secret quarterly)

YubiKey Provisioning

In an air-gapped environment, YubiKey provisioning is done entirely on the internal network through the Vouch server’s web UI. This chapter covers the provisioning workflow, hardware requirements, and spare key strategy.

Provisioning Workflow

1. Administrator creates a user account via the Vouch server web interface
2. User navigates to https://auth.internal on their workstation browser
3. User inserts their YubiKey and completes the WebAuthn registration flow
4. User sets a PIN on their YubiKey if one is not already configured (minimum 8 characters)
5. The credential is registered and the user can begin authenticating

YubiKey Requirements

• YubiKey 5 series with firmware 5.2+
• FIDO2/WebAuthn support enabled
• PIN configured (minimum 8 characters)

Spare Key Strategy

Each user should register at least two YubiKeys (primary and backup). If a YubiKey is lost or damaged:

1. User reports lost key to administrator
2. Administrator revokes the lost key’s credential via the web UI
3. User registers their backup YubiKey

Operations

This chapter covers the day-to-day operational procedures for maintaining a Vouch deployment in an air-gapped environment, including time synchronization, software updates, audit log export, disaster recovery, and troubleshooting.

Time Synchronization

Certificate validity depends on accurate time. Options for air-gapped networks:

GPS Time Receiver (Recommended)

+----------------+     +--------------------+
| GPS Receiver   |---->| Internal NTP       |
| (one-way data) |     | Server (stratum 1) |
+----------------+     +--------------------+
         |                      |
         |                      v
    One-way only         All internal hosts
    (no data out)

Configure NTP clients:

# /etc/ntp.conf
server ntp.internal iburst

Manual Time Sync

For truly isolated networks without GPS:

1. Reference time from secure source (atomic clock, verified external)
2. Set time on NTP server manually
3. Document time sync in audit log

Vouch server configuration is done via environment variables (see Configure Vouch Server). JWT clock skew tolerance is handled automatically.

Software Updates

Update Procedure

1. Download updated packages (connected environment)

# Download latest packages from packages.vouch.sh
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-server-1.1.0-1.x86_64.rpm
curl -LO https://packages.vouch.sh/rpm/x86_64/vouch-1.1.0-1.x86_64.rpm

# For container deployments
docker pull ghcr.io/vouch-sh/vouch:1.1.0
docker save ghcr.io/vouch-sh/vouch:1.1.0 -o vouch-server-1.1.0.tar

2. Verify signatures (connected environment)

rpm -K vouch-server-1.1.0-1.x86_64.rpm
rpm -K vouch-1.1.0-1.x86_64.rpm

3. Transfer via approved media (sneakernet)

4. Verify again (air-gapped environment)

rpm -K vouch-server-1.1.0-1.x86_64.rpm
sha256sum -c SHA256SUMS

5. Apply update

For RPM installations:

# Backup database before upgrade
cp /data/vouch.db /data/vouch.db.backup.$(date +%Y%m%d)

# Upgrade package (migrations run automatically on next startup)
rpm -Uvh vouch-server-1.1.0-1.x86_64.rpm

# Restart service
systemctl restart vouch-server

# Verify health
curl -k https://auth.internal/health

For container deployments:

docker load < vouch-server-1.1.0.tar
# Update docker-compose.yml image tag, then:
docker-compose up -d

Rollback

For RPM installations:

# Restore database backup
cp /data/vouch.db.backup.YYYYMMDD /data/vouch.db

# Downgrade package
rpm -Uvh --oldpackage vouch-server-1.0.0-1.x86_64.rpm

# Restart service
systemctl restart vouch-server

Audit Log Export

Air-gapped environments still need audit trails for compliance.

One-Way Data Diode

+-----------------+     +-------------+     +-----------------+
| Air-Gapped      |---->| Data Diode  |---->| SIEM            |
| Vouch Server    |     | (hardware)  |     | (connected)     |
|                 |     |             |     |                 |
| UDP syslog out  |     | One-way     |     | Splunk/Datadog  |
+-----------------+     +-------------+     +-----------------+

Syslog export is planned but not yet implemented. Currently, use the periodic export method below.

Periodic Export

#!/bin/bash
# Weekly audit log export script

DATE=$(date +%Y%m%d)
OUTPUT_DIR=/mnt/export

# Export audit logs from SQLite directly
sqlite3 /data/vouch.db \
  ".mode json" \
  "SELECT * FROM auth_events WHERE created_at >= datetime('now', '-7 days');" \
  > $OUTPUT_DIR/audit-$DATE.json

# Encrypt for transport
gpg --encrypt --recipient auditor@company.com \
  $OUTPUT_DIR/audit-$DATE.json

# Generate checksum
sha256sum $OUTPUT_DIR/audit-$DATE.json.gpg > $OUTPUT_DIR/audit-$DATE.sha256

# Remove unencrypted
rm $OUTPUT_DIR/audit-$DATE.json

echo "Export complete: audit-$DATE.json.gpg"

Transfer encrypted exports via approved media to connected compliance systems.

Disaster Recovery

Backup Strategy

Recovery Procedure

1. Stop the service

systemctl stop vouch-server

2. Restore database from backup

cp /data/vouch.db.backup.YYYYMMDD /data/vouch.db
chown vouch:vouch /data/vouch.db

3. Re-sync time

# Verify NTP synchronization
timedatectl status
chronyc tracking  # or ntpq -p

4. Start and validate

systemctl start vouch-server
curl -k https://auth.internal/health

CA Key Recovery

If CA keys are lost, all issued certificates become unverifiable.

Prevention:

• Store CA keys in HSM with backup
• Use split-knowledge for key recovery
• Document key ceremony procedures

Recovery:

1. Generate new CA from backup
2. Re-provision all user credentials
3. Redistribute new CA public key
4. Update all SSH server trust anchors

Security Considerations

Network Segmentation

+-------------------------------------------------------------+
|                    Air-Gapped Network                        |
|                                                              |
|  +-----------------+        +-----------------------------+  |
|  |   Management    |        |      User Network           |  |
|  |   VLAN          |        |                             |  |
|  |                 |        |  +-------+  +-----------+   |  |
|  |  * Vouch Server |<------>|  |Workst.|  | Protected |   |  |
|  |  * SQLite       |        |  +-------+  | Resources |   |  |
|  |                 |        |             +-----------+   |  |
|  +-----------------+        +-----------------------------+  |
|           |                                                  |
|           | Restricted                                       |
|           v                                                  |
|  +-----------------+                                         |
|  | Admin Jumpbox   | <-- Physical access control             |
|  +-----------------+                                         |
+--------------------------------------------------------------+

Physical Security

• Server room access controls
• YubiKey storage procedures
• Media transfer protocols
• Tamper-evident logging

Compliance Mapping

Troubleshooting

Cannot Connect to Vouch Server

# Check network connectivity
ping auth.internal

# Verify TLS
openssl s_client -connect auth.internal:443 -CAfile /etc/vouch/root-ca.crt

# Check server logs (systemd)
journalctl -u vouch-server --since "1 hour ago"

# Check server logs (Docker)
docker-compose logs vouch-server

Certificate Validation Failures

# Check system time
date
timedatectl status

# Verify CA is trusted
ssh-keygen -L -f /path/to/cert  # View certificate details

# Check certificate dates
ssh-keygen -L -f /path/to/cert | grep Valid

YubiKey Not Recognized

# Check USB connection
lsusb | grep Yubico

# Verify FIDO2 functionality
ykman fido info

# Reset FIDO2 application (destructive - re-enrollment required)
ykman fido reset

Environment Variables

All Vouch server configuration is done via environment variables (prefixed with VOUCH_). These can also be passed as command-line arguments.

Core Configuration

Network

Upstream Identity Provider

Configure one upstream IdP for enrollment: either OIDC or SAML 2.0. They are mutually exclusive — if both are configured, the server will refuse to start.

OIDC (External IdP)

These variables configure an external OpenID Connect identity provider for enrollment. All three must be set together for OIDC enrollment to work. At startup, the server fetches the OIDC discovery document from {issuer}/.well-known/openid-configuration to auto-discover authorization, token, and JWKS endpoints.

SAML (External IdP)

These variables configure an external SAML 2.0 identity provider for enrollment. VOUCH_SAML_IDP_METADATA_URL is required for SAML; the others are optional.

Session

SSH CA

OIDC Signing

AWS KMS

DPoP

Cleanup & Retention

CORS

GitHub App

These variables configure the Vouch GitHub App integration for issuing GitHub tokens. The App ID, name, and key are required together for GitHub App functionality. OAuth client ID and secret are additionally needed for GitHub user authentication.

TLS

When both VOUCH_TLS_CERT and VOUCH_TLS_KEY are set, the server listens on port 443 (HTTPS) with an automatic HTTP-to-HTTPS redirect on port 80. The VOUCH_LISTEN_ADDR setting is ignored when TLS is configured.

S3 Configuration

Vouch supports loading configuration from an S3 object for centralized management. S3 configuration values override environment variables.

JWT Assertion

CLI Download URLs

These optional variables configure download links displayed in the server UI.

S3 Configuration Schema

For production deployments, Vouch supports loading configuration from an S3 object. This enables:

• Centralized management — Single source of truth for multi-instance deployments
• Dynamic updates — Configuration changes without server restart (for supported fields)
• TLS hot-reload — Automatic certificate rotation without downtime
• Secrets management — Leverage S3 server-side encryption and IAM for credential protection

Enabling S3 Configuration

Set the following environment variables to enable S3-based configuration:

# Required: bucket name
VOUCH_S3_CONFIG_BUCKET=my-bucket

# Optional: object key (default: config/vouch-server.json)
VOUCH_S3_CONFIG_KEY=config/vouch-server.json

# Optional: AWS region (uses default credential chain region if not set)
VOUCH_S3_CONFIG_REGION=us-west-2

# Optional: polling interval in seconds (default: 60)
VOUCH_S3_CONFIG_POLL_INTERVAL=60

When S3 configuration is enabled, it overrides environment variables. This allows for centralized configuration management with dynamic updates.

JSON Schema

The S3 configuration file is a JSON document with the following schema:

{
  "version": 1,
  "listen_addr": "0.0.0.0:443",
  "rp_id": "vouch.example.com",
  "rp_name": "Example Corp",
  "base_url": "https://vouch.example.com",
  "database_url": "postgres://...",
  "dsql_endpoints": {
    "us-east-1": "postgres://vouch@abc123.dsql.us-east-1.on.aws/postgres"
  },
  "jwt_secret": "32+ character secret",
  "session_hours": 8,
  "org_name": "Example Corp",
  "tls": {
    "cert": "<base64-encoded PEM certificate>",
    "key": "<base64-encoded PEM private key>"
  },
  "oidc": {
    "issuer_url": "https://accounts.google.com",
    "client_id": "...",
    "client_secret": "..."
  },
  "saml": {
    "idp_metadata_url": "https://idp.example.com/saml/metadata",
    "sp_entity_id": "https://vouch.example.com",
    "email_attribute": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
    "domain_attribute": "department"
  },
  "allowed_domains": ["example.com"],
  "ssh_ca_key": "<base64-encoded PEM Ed25519 private key>",
  "ssh_ca_kms_key_id": "mrk-1234abcd5678efgh",
  "oidc_signing_key": "<base64-encoded PEM EC P-256 private key>",
  "oidc_signing_kms_key_id": "mrk-abcd1234efgh5678",
  "oidc_rsa_signing_key": "<base64-encoded PEM RSA-3072 private key>",
  "oidc_rsa_signing_kms_key_id": "mrk-rsa1234abcd5678",
  "jwt_hmac_kms_key_id": "mrk-5678abcd1234efgh",
  "document_key": {
    "kms_key_id": "mrk-<key-id>",
    "encrypted_private_key": "<base64-encoded KMS ciphertext>"
  },
  "dpop": {
    "max_age_seconds": 300
  },
  "cors_origins": ["https://app.example.com"],
  "github": {
    "app_id": 12345,
    "app_name": "my-vouch-app",
    "app_key": "<PEM RSA private key>",
    "webhook_secret": "<secret>",
    "client_id": "<oauth-client-id>",
    "client_secret": "<oauth-client-secret>"
  },
  "cleanup_interval_minutes": 15,
  "auth_events_retention_days": 90,
  "oauth_events_retention_days": 90,
  "cli_download_macos": "https://example.com/vouch-macos",
  "cli_download_linux": "https://example.com/vouch-linux",
  "cli_download_windows": "https://example.com/vouch-windows",
  "device_code_expires_seconds": 600,
  "device_poll_interval_seconds": 5
}

Field Descriptions

Base64 Encoding

All certificate and key fields in the S3 configuration must be base64-encoded PEM strings. To encode a PEM file:

# Encode a PEM file for S3 config
base64 -i cert.pem | tr -d '\n'

This ensures proper handling of newlines and special characters within JSON values.

Hot-Reloadable vs Startup-Only Fields

The server polls S3 at the configured interval and detects changes via ETag comparison. However, only certain fields support hot-reload without a server restart:

Non-hot-reloadable fields include: jwt_secret, database_url, listen_addr, rp_id, rp_name, session_hours, cors_origins, allowed_domains, dpop.*, OIDC settings, SAML settings, GitHub App settings, SSH CA key, OIDC signing keys, and all KMS key IDs.

Note: The oidc and saml blocks are mutually exclusive. If both are present, the server will refuse to start.

Changes to non-hot-reloadable fields in S3 are silently ignored. A server restart is required to apply them.

TLS Certificate Hot-Reload

Vouch supports automatic TLS certificate reloading without dropping connections:

1. Via S3 polling — Update tls.cert and tls.key in the S3 config; the server detects the change via ETag and reloads automatically.
2. Via SIGHUP — Send SIGHUP to the server process to reload TLS certificates.

# Manual TLS certificate reload (Unix only)
kill -SIGHUP $(pgrep vouch-server)

Note: SIGHUP only reloads TLS certificates. It does not reload any other configuration fields.

Compliance Mapping

This page maps Vouch features to common compliance frameworks. Vouch’s hardware-backed authentication model satisfies many of the strictest access control requirements across multiple regulatory standards.

NIST 800-53

SOC 2

FedRAMP

HIPAA