> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qwedai.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# Self-hosting

> Run QWED on your own infrastructure using Docker Compose. Quick start guide with services overview including API, PostgreSQL, Redis, and Grafana monitoring.

## Quick start

### Prerequisites

* Docker with the Compose plugin (`docker compose`)
* A clone of the `qwed-verification` repository

```bash theme={null}
# Clone the repository
git clone https://github.com/QWED-AI/qwed-verification.git
cd qwed-verification

# Copy the settings template, then fill in every required value
cp deploy/.env.example deploy/.env

# Build the server image and start the stack from the repository root
docker compose -f deploy/docker-compose.yml up -d

# Check status
docker compose -f deploy/docker-compose.yml ps
```

Compose builds the API server from the repository's `Dockerfile` (the `server` target) and waits for Postgres, Redis, and Jaeger to report healthy before it starts the API.

> [!WARNING]
>
> Compose refuses to start while `QWED_CORS_ORIGINS`, `API_KEY_SECRET`, `QWED_JWT_SECRET_KEY`, `QWED_API_KEY_LOOKUP_SECRET`, `QWED_AUDIT_SECRET_KEY`, or `DATABASE_URL` is empty. The Postgres container refuses to start while `POSTGRES_PASSWORD` is empty, and the Grafana container refuses to start while `GF_SECURITY_ADMIN_PASSWORD` is empty.

## Services

| Service | Port | Purpose |
| - | - | - |
| QWED API (`qwed-core`) | 8000 | Main API |
| PostgreSQL | 5432 | Database |
| Redis | 6379 | Cache & rate limiting |
| Grafana | 3000 | Dashboards, with Prometheus and Jaeger data sources provisioned |
| Prometheus | 9090 | Metrics. Scrapes the API at `qwed-core:8000` |
| Jaeger | 16686 | Tracing |

The API server keeps its data in the `qwed-data` volume. Postgres, Redis, Prometheus, and Grafana each have their own named volume, so data survives `docker compose down`.

## Configuration

### Environment variables

`deploy/.env.example` lists every setting the Compose stack reads. Generate each secret separately:

```bash theme={null}
python -c "import secrets; print(secrets.token_urlsafe(48))"
```

```bash theme={null}
# deploy/.env

# QWED API server (required)
QWED_CORS_ORIGINS=https://app.yourcompany.com
# Generate each secret separately and keep it:
#   python -c "import secrets; print(secrets.token_urlsafe(48))"
API_KEY_SECRET=
# Changing them invalidates issued tokens and API keys, or breaks the signed
# audit chain. The lookup secret must differ from the JWT secret.
QWED_JWT_SECRET_KEY=
QWED_API_KEY_LOOKUP_SECRET=
QWED_AUDIT_SECRET_KEY=

# Optional. User IDs allowed to read /metrics. Empty denies every caller.
QWED_METRICS_OPERATOR_USER_IDS=

# Postgres (required)
POSTGRES_USER=qwed
POSTGRES_DB=qwed_db
POSTGRES_PASSWORD=
# Must match the three values above. Build it as a postgresql:// URL with the
# user, POSTGRES_PASSWORD, the host "postgres" (the Compose service), port 5432
# and qwed_db, the same shape as deploy/.env.example shows.
DATABASE_URL=

# Grafana (required)
GF_SECURITY_ADMIN_USER=admin
# Generate a separate password the same way:
#   python -c "import secrets; print(secrets.token_urlsafe(24))"
GF_SECURITY_ADMIN_PASSWORD=
```

Postgres and Grafana receive only their own settings, not the API server's secrets. `deploy/.env` is git-ignored.

> [!WARNING]
>
> `QWED_METRICS_OPERATOR_USER_IDS` is optional. Leave it empty and every caller, including the Prometheus scraper, gets denied on `/metrics`. Prometheus records `401` responses until you configure an operator. See [Prometheus metrics](#prometheus-metrics). See the [deployment guide](https://docs.qwedai.tech/advanced/deployment.md#environment-variables-reference) for the full variable reference.

### API configuration

```python theme={null}
# config.py
SETTINGS = {
    "rate_limit_enabled": True,
    "rate_limit_per_minute": 60,
    "cache_ttl_seconds": 3600,
    "max_query_length": 10000,
    "allowed_engines": ["math", "logic", "code", "sql"],
}
```

## Docker

### API server image

The `qwedai/qwed-verification` image on Docker Hub runs the API server with uvicorn on port 8000. Docker Hub publishes `latest` from `main` and a versioned tag for each release from 8.0.0.

The image:

* Runs as a non-root user (UID 1000).
* Uses `/var/lib/qwed` as its working directory. The default SQLite database (`sqlite:///./qwed.db`) lives there. Mount a volume at `/var/lib/qwed` to keep it, or set `DATABASE_URL` to use Postgres.
* Has a health check that calls `GET /health` with the image's own Python.
* Reads its bind address and port from `UVICORN_HOST` (default `0.0.0.0`) and `UVICORN_PORT` (default `8000`). The health check follows `UVICORN_PORT`, so change the port with `-e UVICORN_PORT=...` rather than a `--port` argument.

> [!WARNING]
>
> Tags before 8.0.0 ran the QWED GitHub Action, not the API server. If you ran the image as a repository scanner, use the [GitHub Action](https://docs.qwedai.tech/advanced/github-action.md) instead.

### Run the published image

Pass the secrets from your shell, so no value is written into the command:

```bash theme={null}
# Generate each secret once and keep it. Changing the lookup or audit key later
# invalidates existing API keys or audit signatures.
for name in API_KEY_SECRET QWED_JWT_SECRET_KEY QWED_API_KEY_LOOKUP_SECRET QWED_AUDIT_SECRET_KEY; do
  export "$name=$(python -c 'import secrets; print(secrets.token_urlsafe(48))')"
done

docker run -d -p 8000:8000 \
  -v qwed-data:/var/lib/qwed \
  -e QWED_CORS_ORIGINS="https://app.yourcompany.com" \
  -e API_KEY_SECRET \
  -e QWED_JWT_SECRET_KEY \
  -e QWED_API_KEY_LOOKUP_SECRET \
  -e QWED_AUDIT_SECRET_KEY \
  qwedai/qwed-verification:9.1.0
```

The server refuses to start without any of `QWED_CORS_ORIGINS`, `API_KEY_SECRET`, `QWED_JWT_SECRET_KEY`, `QWED_API_KEY_LOOKUP_SECRET` and `QWED_AUDIT_SECRET_KEY`, and `QWED_API_KEY_LOOKUP_SECRET` must differ from `QWED_JWT_SECRET_KEY`. Also set `QWED_SIGNING_KEY` (an EC P-256 key), or attestations stop verifying after a restart or on another replica.

The server names no model. For requests that use a provider (natural-language, logic, statistics, and fact), also pass the provider's settings, for example `-e ACTIVE_PROVIDER -e OPENAI_API_KEY -e OPENAI_MODEL`. The container doesn't read the `qwed init` settings on your host. If you keep the values in a `.env` file instead, pass it with `--env-file .env`.

Optionally, add a signing key so attestations verify after a restart and on every replica. Without one, each process generates its own key. Mount an unencrypted EC P-256 PEM file and point `QWED_SIGNING_KEY_FILE` at it by adding these flags to `docker run`:

```bash theme={null}
  -v "$PWD/attestation-key.pem:/run/secrets/attestation-key.pem:ro" \
  -e QWED_SIGNING_KEY_FILE=/run/secrets/attestation-key.pem \
```

See [Signing keys](https://docs.qwedai.tech/advanced/attestations.md#signing-keys) and the [environment variables reference](https://docs.qwedai.tech/advanced/deployment.md#environment-variables-reference).

To serve on a different port, with the same secrets exported:

```bash theme={null}
docker run -d -p 9000:9000 -e UVICORN_PORT=9000 \
  -v qwed-data:/var/lib/qwed \
  -e QWED_CORS_ORIGINS="https://app.yourcompany.com" \
  -e API_KEY_SECRET \
  -e QWED_JWT_SECRET_KEY \
  -e QWED_API_KEY_LOOKUP_SECRET \
  -e QWED_AUDIT_SECRET_KEY \
  qwedai/qwed-verification:9.1.0
```

### Build from source

The repository's `Dockerfile` has two targets. `server` is the API server. `action` is the GitHub Action and stays the default target, so build the server with `--target server`:

```bash theme={null}
docker build --target server -t qwed-api .
```

## Kubernetes

The repository ships manifests in `deploy/kubernetes/`. The Deployment pins `docker.io/qwedai/qwed-verification:9.1.0`. Tags from 8.0.0 contain the API server. Fill in every value in `deploy/kubernetes/secret.yaml` before you apply it. See [Kubernetes deployment](https://docs.qwedai.tech/advanced/deployment.md#kubernetes-deployment).

```yaml theme={null}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: qwed-core
  namespace: qwed
spec:
  replicas: 1
  selector:
    matchLabels:
      app: qwed-core
  template:
    metadata:
      labels:
        app: qwed-core
    spec:
      containers:
      - name: qwed-core
        image: docker.io/qwedai/qwed-verification:9.1.0
        ports:
        - containerPort: 8000
        envFrom:
        - configMapRef:
            name: qwed-config
        - secretRef:
            name: qwed-secrets
```

> 🏢 **Enterprise Support Coming Soon:** Managed hosting, dedicated support, and SLA guarantees. Contact [support@qwedai.tech](mailto:support@qwedai.tech)

## Scaling

### Horizontal scaling

* Stateless API servers behind load balancer
* Redis for distributed rate limiting
* PostgreSQL with read replicas

### Performance tuning

```bash theme={null}
# Enable connection pooling
export QWED_DB_POOL_SIZE=20
```

> [!WARNING]
>
> Run one uvicorn worker per database. Multiple workers or replicas writing audit rows to the same database are not supported. Every verification request appends an audit row, and the audit logger keeps each organization's chain head in memory. With `--workers 4` or several replicas, a conflicting append makes the verification request fail with `500 audit persistence failed` instead of forking the chain. See [Chain continuity across restarts](https://docs.qwedai.tech/advanced/compliance.md#chain-continuity-across-restarts).

### Availability limits

QWED bounds request size, slow work, and sandbox usage so one request can't stall or exhaust a server process. The defaults suit most deployments. Raise them only if your host has the headroom.

| Variable | Default | What it bounds |
| - | - | - |
| `QWED_CODE_VERIFY_TIMEOUT` | `10` | Seconds a `/verify/code` scan can run. A timeout returns `503`. |
| `QWED_PROVIDER_TIMEOUT` | `30` | Seconds an LLM provider call can run for `/verify/natural_language`, `/verify/logic`, and `/verify/fact`. Calls run off the event loop. A timeout fails closed: `/verify/fact` returns `503`, the others return an `ERROR` result. |
| `QWED_Z3_TIMEOUT` | `15` | Seconds a `/verify/logic` Z3 solve can run. Solves run one at a time on a dedicated thread. |
| `QWED_BATCH_MAX_JOBS` | `1000` | Jobs kept in the `/verify/batch` job store. QWED evicts the oldest finished job first and never evicts a running job. An evicted job returns `404` from `GET /verify/batch/{job_id}`. When every stored job is active, new submissions get `503 Batch capacity reached — retry shortly.` |
| `QWED_BATCH_ITEM_TIMEOUT` | `60` | Seconds a single batch item can run. A timed-out item is marked `failed` so the job can finish. |
| `QWED_SANDBOX_MAX_CONCURRENT` | `4` | Sandbox containers running at once per server process, across `/verify/consensus` and `/verify/stats` with the Docker executor. |
| `QWED_SANDBOX_ACQUIRE_TIMEOUT` | `10` | Seconds a request waits for a free sandbox slot. If none frees in time, QWED rejects the run with `Sandbox capacity reached — retry shortly.` instead of starting another container. |

```bash theme={null}
# Example: a larger host that runs more sandboxed verifications in parallel
export QWED_SANDBOX_MAX_CONCURRENT=8
export QWED_PROVIDER_TIMEOUT=45
```

Request body caps are fixed. Every `/verify` JSON route accepts up to 1,000,000 bytes. `/verify/stats` and `/verify/image` accept their 10 MB file limit plus envelope headroom. QWED rejects an oversized body with `413` before processing it. See [Request limits](https://docs.qwedai.tech/api/endpoints.md#request-limits) for the full error list.

> [!NOTE]
>
> `QWED_SANDBOX_MAX_CONCURRENT` applies per worker process. With `uvicorn --workers 4` and the default of 4, a host can run up to 16 sandbox containers. Size the value against your host's memory, CPU, and PID limits.

### Sandbox image and container user

> [!NOTE]
>
> v8.0.0

`/verify/stats` and `/verify/consensus` run model-generated Python in a Docker sandbox. QWED pins the sandbox image by digest, so the sandbox only ever runs the exact image that was reviewed:

```text theme={null}
amancevice/pandas@sha256:9257b88909bddf60c8a85b06d572fc6dfc708ea9c8a3c3dc2149e312d57f91e3
```

Each sandbox container runs with a read-only root filesystem, drops all Linux capabilities, sets `no-new-privileges`, and never runs as a privileged root. The only writable path is the per-execution `/workspace` directory, which is a private `0700` host directory. The container runs as that directory's owner:

* **Non-root server process:** the container runs as the host user.
* **Root server process:** QWED hands each workspace to a fresh uid for that execution. QWED skips any uid that a host account or group owns, that `/etc/subuid` or `/etc/subgid` delegates, or that another running execution uses.
* **Rootless daemon run by the same non-root user:** the container runs as its root, which the daemon maps to that unprivileged host user.

| Variable | Default | Purpose |
| - | - | - |
| `QWED_SANDBOX_IMAGE` | Pinned image above | Runs a different sandbox image, such as a mirror in your own registry. The value must be pinned by digest (`name@sha256:<64 hex>`). QWED refuses any other value, logs an error, and uses the pinned image. |
| `QWED_SANDBOX_UID_RANGE` | `1879048192-2147483646` | The uid range QWED draws workspace owners from when the server process runs as root. Use `FIRST-LAST` within `65536`–`2147483646`. QWED logs an invalid value and uses the default. The default is a range systemd documents as unused. |

```bash theme={null}
# Mirror the sandbox image in a private registry, pinned by digest
export QWED_SANDBOX_IMAGE=registry.example.com/qwed/pandas@sha256:9257b88909bddf60c8a85b06d572fc6dfc708ea9c8a3c3dc2149e312d57f91e3

# Reserve a dedicated uid range for root-run deployments
export QWED_SANDBOX_UID_RANGE=2000000000-2000099999
```

> [!WARNING]
>
> QWED never guesses how the Docker daemon maps users. It reports the sandbox as unavailable (`SECURE_RUNTIME_UNAVAILABLE`) when any of the following is true. `/verify/stats` then returns `BLOCKED` with constraint `stats_verifier.runtime_unavailable`, and `/verify/consensus` counts its code-execution engine as `BLOCKED`.
>
> * the daemon uses `userns-remap`
> * the server process runs as root with a rootless daemon
> * QWED can't read the daemon's security options

## Monitoring

### Health check

```bash theme={null}
curl http://localhost:8000/health
```

### Prometheus metrics

```text theme={null}
qwed_verification_total{engine="math", status="verified"}
qwed_verification_latency_seconds
qwed_cache_hit_rate
```

Scrape `GET /metrics/prometheus` (the Prometheus text exposition format — `/metrics` returns JSON). The endpoint requires a platform operator: create a dedicated scraper account, add its user ID to `QWED_METRICS_OPERATOR_USER_IDS` (fail-closed when unset — every caller is denied, including org owners/admins), mint the account an API key, and send it as a header on the scrape. See [All-tenant metrics authorization](https://docs.qwedai.tech/advanced/security-hardening.md#all-tenant-metrics-authorization).

```yaml theme={null}
# prometheus scrape config
scrape_configs:
  - job_name: 'qwed-api'
    metrics_path: '/metrics/prometheus'
    static_configs:
      - targets: ['localhost:8000']
    http_headers:
      X-Api-Key:
        values: ['qwed_live_********************************wDHo']  # masked v2 example (see API key format) — render the real one from your secret pipeline
```

## Security

1. **Always use HTTPS** in production
2. **Set `API_KEY_SECRET`** — mandatory, no default value
3. **Set `QWED_CORS_ORIGINS`** — mandatory, explicitly list allowed origins
4. **Set strong API keys**
5. **Enable rate limiting**
6. **Use network isolation**
7. **Rotate secrets regularly**
8. **Configure `QWED_METRICS_OPERATOR_USER_IDS`** if anything needs all-tenant metrics — unset means deny-all


## Related topics

- [QWED Security GitHub App: deterministic PR verification](https://docs.qwedai.tech/advanced/github-app.md)
- [QWED architecture: translation, verification, guards](https://docs.qwedai.tech/architecture.md)
- [Installation](https://docs.qwedai.tech/getting-started/installation.md)
- [LLM configuration](https://docs.qwedai.tech/getting-started/llm-configuration.md)
- [Custom providers](https://docs.qwedai.tech/getting-started/custom-providers.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.