> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qwedai.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM configuration

> Configure QWED with any LLM provider — OpenAI, Anthropic, Gemini, Azure, Ollama, or custom OpenAI-compatible endpoints — using portable YAML.

QWED works with any LLM provider. The fastest way to get configured is the onboarding command:

```bash theme={null}
qwed init
```

This verifies your verification engines are operational, walks you through provider selection and secure key entry, and bootstraps a local API key — all in a single command. For CI/CD pipelines, pass `--non-interactive` with `--provider` and `--api-key` flags. See the [CLI reference](https://docs.qwedai.tech/advanced/cli.md) for the full walkthrough.

The rest of this page covers manual configuration and advanced options.

## Understanding QWED's architecture

QWED uses LLMs as **untrusted translators**, not as answer generators:

```
┌──────────────────────────────────────────────────────────────────┐
│                    QWED VERIFICATION PIPELINE                     │
├──────────────────────────────────────────────────────────────────┤
│                                                                   │
│  User Query     LLM (Translator)     QWED (Verifier)    Result   │
│      │                │                    │               │     │
│      ▼                ▼                    ▼               ▼     │
│  "Is 2+2=5?"  →  "2+2=5"  →  [SymPy: 2+2=4]  →  ❌ CORRECTED: 4  │
│                                                                   │
│  Your LLM           ↑            Our Engines      Deterministic  │
│  (any provider)   Untrusted        (Trusted)       Guarantee     │
│                                                                   │
└──────────────────────────────────────────────────────────────────┘
```

> **Key Insight:** The LLM translates natural language to structured form. QWED then verifies the structured form using deterministic engines. The LLM can be wrong — QWED catches and corrects errors.

## Supported LLM providers

| Provider | Env variable | Model variable | `qwed init` support |
| - | - | - | - |
| **NVIDIA NIM** | `CUSTOM_API_KEY` + `CUSTOM_BASE_URL` | `CUSTOM_MODEL` | Yes |
| **OpenAI** | `OPENAI_API_KEY` | `OPENAI_MODEL` | Yes |
| **Anthropic** | `ANTHROPIC_API_KEY` | `ANTHROPIC_MODEL` | Yes |
| **Google Gemini** | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | `GEMINI_MODEL` | Yes |
| **Ollama (local)** | `OLLAMA_BASE_URL` | `OLLAMA_MODEL` | Manual |
| **OpenAI-compatible** | `CUSTOM_BASE_URL` + `CUSTOM_API_KEY` | `CUSTOM_MODEL` | Yes |
| **Azure OpenAI** | `AZURE_OPENAI_ENDPOINT` + `AZURE_OPENAI_API_KEY` | `AZURE_OPENAI_DEPLOYMENT` | Manual |
| **Claude Opus** | `CLAUDE_OPUS_API_KEY` | `CLAUDE_OPUS_DEPLOYMENT` | Manual |
| **Custom (YAML)** | Defined per provider | `CUSTOM_MODEL` | Manual (choose Custom Provider and enter the endpoint and model) |

> [!WARNING]
>
> From v9.0.0, QWED names no default model. Set the model variable of the provider you use. QWED sends the value to the provider unchanged, so any model or deployment ID the provider serves works. When no model is set, the provider raises an error that names the variable to set. Nothing falls back to another model. On the server, `ANTHROPIC_DEPLOYMENT` still wins over `ANTHROPIC_MODEL` when it is set.

> [!TIP]
>
> You can add any OpenAI-compatible provider (Groq, Together, Fireworks, and others) using a YAML configuration file. See [Custom providers](https://docs.qwedai.tech/getting-started/custom-providers.md) for details.

***

## Configuration options

### Option 1: use QWED's built-in translation (recommended)

QWED can handle LLM translation internally:

```python theme={null}
from qwed import QWEDClient

# QWED uses its own LLM for translation
client = QWEDClient(api_key="qwed_your_key")

result = client.verify("What is 15% of 200?")
# QWED internally: "15% of 200" → 0.15 * 200 → verify with SymPy → 30
```

### Option 2: bring your own LLM

Use QWED purely as a verification layer:

```python theme={null}
from qwed import QWEDClient
import openai

# Your LLM call
openai_client = openai.OpenAI(api_key="sk-...")
response = openai_client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is 847 × 23?"}]
)
llm_answer = response.choices[0].message.content

# QWED verification only (no LLM needed)
qwed = QWEDClient(api_key="qwed_your_key")
result = qwed.verify_math(
    expression="847 * 23",
    expected_result=llm_answer
)

if result.verified:
    print(f"✅ LLM was correct: {llm_answer}")
else:
    print(f"❌ LLM was wrong. Correct: {result.corrected}")
```

### Option 3: Self-hosted with custom LLM

For self-hosted deployments, run `qwed init` or manually create a `.env` file:

```bash theme={null}
# .env file — generated by `qwed init` or created manually
# WARNING: Contains secrets. NEVER commit this file.

# Active provider (openai, openai_direct, openai_compat, azure_openai,
# anthropic, claude_opus, gemini, ollama, auto)
ACTIVE_PROVIDER=openai

# Set the model variable for your provider. QWED names no default model.

# OpenAI
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o-mini

# OR Anthropic
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_MODEL=claude-sonnet-4-20250514

# OR Ollama (no key needed)
# OLLAMA_BASE_URL=http://localhost:11434/v1
# OLLAMA_MODEL=llama3

# OR Google Gemini
# GOOGLE_API_KEY=your-google-api-key
# GEMINI_MODEL=your-gemini-model

# OR OpenAI-compatible endpoint (Groq, Together, LM Studio, etc.)
# CUSTOM_BASE_URL=https://inference.do-ai.run/v1
# CUSTOM_API_KEY=your-key
# CUSTOM_MODEL=gpt-4o-mini
```

> [!NOTE]
>
> When `ACTIVE_PROVIDER` is not set, QWED defaults to **Ollama** as a safe local fallback. Set `ACTIVE_PROVIDER` explicitly if you want to use a cloud provider.
>
> An `ACTIVE_PROVIDER` that names no provider QWED knows stops the request with an error naming `ACTIVE_PROVIDER`. A request that names its own provider must name one QWED knows, or it is refused. Each request runs on the provider it names, else on the one `ACTIVE_PROVIDER` selects.

QWED loads `.env` files in a specific order: the project-level `.env` (in the current working directory) takes precedence, followed by the global `~/.qwed/.env`. This ensures your project-specific configuration always overrides global defaults. If `python-dotenv` is not installed, `.env` loading is skipped gracefully with a warning.

***

## Provider-specific setup

> [!TIP]
>
> You can configure any of these providers interactively by running `qwed init`.

### OpenAI

```bash theme={null}
export ACTIVE_PROVIDER=openai
export OPENAI_API_KEY=sk-your-key-here
export OPENAI_MODEL=gpt-4o-mini
```

Key format: `sk-...` or `sk-proj-...` — get yours at [platform.openai.com/api-keys](https://platform.openai.com/api-keys).

### Anthropic (Claude)

```bash theme={null}
export ACTIVE_PROVIDER=anthropic
export ANTHROPIC_API_KEY=sk-ant-your-key-here
export ANTHROPIC_MODEL=your-claude-model
```

Key format: `sk-ant-...` — get yours at [console.anthropic.com/settings/keys](https://console.anthropic.com/settings/keys).

The server's Anthropic provider is an Azure AI Foundry client: set `ANTHROPIC_ENDPOINT` to your Foundry endpoint in the server's environment. On the server, `ANTHROPIC_ENDPOINT` and `ANTHROPIC_API_KEY` are both required, or the provider stops with `Missing Anthropic environment variables`. The `qwed` CLI calls the Anthropic API directly instead: it reads `ANTHROPIC_API_KEY` and `ANTHROPIC_MODEL`, and stops if `ANTHROPIC_ENDPOINT` is set.

The Anthropic provider doesn't send `temperature`, `top_p`, or `top_k`. The anthropic SDK has had no sampling parameters since 1.0, and Anthropic API version `2026-10-21`, which the 2.x SDK sends, refuses them. Claude samples with the model's defaults, and no QWED setting overrides them. Sampling can vary the translations Claude returns and the verdicts `verify_fact` and `verify_image` return. QWED's fact and image verifiers record those verdicts only as advisory checks, never as `VERIFIED`.

### Local LLMs (Ollama)

No API key needed. Install Ollama, pull a model, and configure QWED:

```bash theme={null}
# Install and start Ollama
ollama serve

# Pull a model
ollama pull llama3

# Configure QWED
export ACTIVE_PROVIDER=ollama
export OLLAMA_BASE_URL=http://localhost:11434/v1
export OLLAMA_MODEL=llama3
```

### OpenAI-compatible endpoint

For any service with an OpenAI-compatible API — DigitalOcean, Groq, Together AI, LM Studio, vLLM, and others:

```bash theme={null}
export ACTIVE_PROVIDER=openai_compat
export CUSTOM_BASE_URL=https://inference.do-ai.run/v1
export CUSTOM_API_KEY=your-api-key
export CUSTOM_MODEL=gpt-4o-mini
```

`CUSTOM_BASE_URL` and `CUSTOM_MODEL` are required. If your endpoint does not require authentication (for example, a local vLLM or LM Studio server), you can omit `CUSTOM_API_KEY` entirely. QWED supplies a placeholder token so the underlying client initializes correctly.

### Google Gemini

Gemini uses a native `GeminiProvider` powered by the `google-generativeai` SDK. Install the dependency first:

```bash theme={null}
pip install google-generativeai
```

Then configure your environment:

```bash theme={null}
export ACTIVE_PROVIDER=gemini
export GOOGLE_API_KEY=your-google-api-key
export GEMINI_MODEL=your-gemini-model
```

You can also use `GEMINI_API_KEY` instead of `GOOGLE_API_KEY`. Get yours at [aistudio.google.com/app/apikey](https://aistudio.google.com/app/apikey).

The Gemini provider supports math translation, logic verification, stats query generation, fact verification, and image claim verification. All API calls use a 30-second timeout and `temperature=0.0` for deterministic output. If `google-generativeai` is not installed, QWED returns a structured `ImportError` instead of crashing.

### Azure OpenAI

```bash theme={null}
export ACTIVE_PROVIDER=azure_openai
export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
export AZURE_OPENAI_API_KEY=your-azure-key
export AZURE_OPENAI_DEPLOYMENT=your-deployment-name
export AZURE_OPENAI_API_VERSION=2024-02-15-preview
```

### Claude Opus

```bash theme={null}
export ACTIVE_PROVIDER=claude_opus
export CLAUDE_OPUS_ENDPOINT=https://your-foundry-endpoint
export CLAUDE_OPUS_API_KEY=sk-ant-your-key
export CLAUDE_OPUS_DEPLOYMENT=claude-opus-4-5
```

Like the Anthropic provider, the Claude Opus provider doesn't send `temperature` and samples with the model's defaults. Sampling can vary translations and the verdicts `verify_fact` and `verify_image` return. QWED records those verdicts only as advisory checks, never as `VERIFIED`.

***

## Universal provider config (YAML)

> [!NOTE]
>
> New in v4.0.0

You can define custom LLM providers using a YAML configuration file at `~/.qwed/providers.yaml`. This is useful for managing multiple providers, custom endpoints, or community-contributed provider configs.

### YAML format

```yaml theme={null}
providers:
  my-custom-llm:
    base_url: "https://api.example.com/v1"
    api_key_env: "MY_CUSTOM_API_KEY"
    default_model: "my-model-v2"
    models_endpoint: "/models"
    auth_header: "Authorization"
    auth_prefix: "Bearer"
```

| Field | Required | Default | Description |
| - | - | - | - |
| `base_url` | Yes | — | The provider's API base URL |
| `api_key_env` | Yes | — | Environment variable name containing the API key |
| `default_model` | No | — | Suggested model, recorded with the provider. The provider runs `CUSTOM_MODEL`, so set that variable (or enter the model in the wizard) |
| `models_endpoint` | No | `/models` | Endpoint for listing available models |
| `auth_header` | No | `Authorization` | HTTP header name for authentication |
| `auth_prefix` | No | `Bearer` | Prefix for the auth header value |

> [!WARNING]
>
> The YAML file is written with `0600` permissions (owner-only) for security. The `~/.qwed/` directory is created with `0700` permissions.

### Import community providers

You can import provider configurations from a URL:

```python theme={null}
from qwed_sdk.cli import import_provider

# Import a community-contributed provider config
import_provider("https://example.com/my-provider.yaml")
```

Or use the CLI:

```bash theme={null}
qwed provider import https://example.com/my-provider.yaml
```

Imported providers are validated and sandboxed — only the allowed fields listed above are saved. Provider slugs are sanitized and cannot shadow built-in providers.

### Key validation

QWED validates API keys in two stages:

1. **Format check** — Regex-based pattern matching (no network call). Built-in patterns include `sk-...` for OpenAI and `sk-ant-...` for Anthropic.
2. **Connection test** — Lightweight read-only request to the provider's models endpoint to confirm the key works.

Run `qwed init` to validate your keys interactively, or test programmatically:

```python theme={null}
from qwed_sdk.cli import test_provider_connection

success, message = test_provider_connection("openai")
```

***

## Provider routing

QWED automatically routes queries to the appropriate provider based on your configuration and query content.

### Alias normalization

Provider names are normalized before routing, so common variations like `openai-compatible`, `openai_compatible`, and `openai_compat` all resolve to the same OpenAI-compatible provider. You do not need to worry about exact casing or separators when specifying a provider in API requests or environment variables.

### Content-aware routing

When no preferred provider is specified in a request, QWED uses the configured default. For certain query types, QWED applies content-aware heuristics:

| Query type | Routed to |
| - | - |
| Math/logic keywords (`calculate`, `solve`, `equation`, `proof`) | Your configured default provider |
| Creative/writing keywords (`write`, `compose`, `essay`, `story`) | Anthropic (Claude) |
| All other queries | Your configured default provider |

You can always override routing by passing an explicit `provider` parameter in your API request.

***

## Programmatic configuration

```python theme={null}
from qwed import QWEDClient, LLMConfig

# Option 1: Use environment variables (recommended)
client = QWEDClient()

# Option 2: Explicit configuration
client = QWEDClient(
    api_key="qwed_your_key",
    llm_config=LLMConfig(
        provider="openai",
        api_key="sk-...",
        model="gpt-4o",
        temperature=0.0,
    )
)

# Option 3: No LLM (verification only)
client = QWEDClient(
    api_key="qwed_your_key",
    llm_enabled=False  # Only use deterministic verification
)
```

***

## Translation vs verification

Understanding the two phases:

| Phase | What Happens | Who Does It | Required? |
| - | - | - | - |
| **Translation** | Natural language → Structured form | LLM (any) | Optional |
| **Verification** | Structured form → Proof | QWED Engines | Required |

### When you need an LLM

* `client.verify("Is the derivative of x² equal to 2x?")` — Needs LLM to parse
* `client.verify("Calculate compound interest on $1000 at 5%")` — Needs LLM

### When you don't need an LLM

* `client.verify_math("diff(x**2, x) == 2*x")` — Already structured
* `client.verify_logic("(AND (GT x 5) (LT x 10))")` — Already in DSL
* `client.verify_sql("SELECT * FROM users")` — Already structured
* `client.verify_code("import os; os.system('rm -rf /')")` — Code, not NL

***

## FAQ

### Do I need an LLM to use QWED?

**No.** If you're sending structured queries (math expressions, SQL, code, QWED-Logic DSL), you don't need an LLM. QWED engines work directly on structured input.

### Can I use my own LLM and just use QWED for verification?

**Yes.** This is the "Bring Your Own LLM" pattern. Call your LLM, then pass its output to QWED for verification.

### Which LLM is best for QWED translation?

QWED is model-agnostic and recommends no model. Use any model your provider serves: the deterministic engines check every translation it produces.

### Is the LLM translation deterministic?

No. LLMs are inherently probabilistic. The Anthropic and Claude Opus providers don't send `temperature`, so Claude samples with the model's defaults. Sampling can vary translations and the verdicts `verify_fact` and `verify_image` return. QWED's fact and image verifiers record those verdicts only as advisory checks, never as `VERIFIED`. The OpenAI, Azure OpenAI, OpenAI-compatible, Gemini, and Ollama providers send `temperature=0.0`, which reduces variation but doesn't remove it.

That's why **QWED verification is essential**. The deterministic engines give the same verdict for the same translation. They check the expression the model produced, not whether it matches your question, so two different translations of one question can get different verdicts.

***

## Next steps

* [Quick start tutorial](https://docs.qwedai.tech/getting-started/quickstart.md)
* [Custom providers](https://docs.qwedai.tech/getting-started/custom-providers.md) — add any OpenAI-compatible endpoint via YAML
* [QWED-Logic DSL reference](https://docs.qwedai.tech/api/dsl-reference.md)
* [Self-hosting guide](https://docs.qwedai.tech/advanced/self-hosting.md)


## Related topics

- [Why cloud LLMs for QWED verification?](https://docs.qwedai.tech/advanced/why-cloud-llms.md)
- [QWED: deterministic verification for LLMs and AI agents](https://docs.qwedai.tech/intro.md)
- [QWEDLocal — client-side verification](https://docs.qwedai.tech/advanced/qwed-local.md)
- [Frequently asked questions](https://docs.qwedai.tech/faq.md)
- [QWED CLI reference](https://docs.qwedai.tech/advanced/cli.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.