LLM configuration

Configure QWED with any LLM provider — OpenAI, Anthropic, Gemini, Azure, Ollama, or custom OpenAI-compatible endpoints — using portable YAML.

QWED works with any LLM provider. The fastest way to get configured is the onboarding command:

qwed init

This verifies your verification engines are operational, walks you through provider selection and secure key entry, and bootstraps a local API key — all in a single command. For CI/CD pipelines, pass --non-interactive with --provider and --api-key flags. See the CLI reference for the full walkthrough.

The rest of this page covers manual configuration and advanced options.

​

Understanding QWED’s architecture

QWED uses LLMs as untrusted translators, not as answer generators:

┌──────────────────────────────────────────────────────────────────┐
│                    QWED VERIFICATION PIPELINE                     │
├──────────────────────────────────────────────────────────────────┤
│                                                                   │
│  User Query     LLM (Translator)     QWED (Verifier)    Result   │
│      │                │                    │               │     │
│      ▼                ▼                    ▼               ▼     │
│  "Is 2+2=5?"  →  "2+2=5"  →  [SymPy: 2+2=4]  →  ❌ CORRECTED: 4  │
│                                                                   │
│  Your LLM           ↑            Our Engines      Deterministic  │
│  (any provider)   Untrusted        (Trusted)       Guarantee     │
│                                                                   │
└──────────────────────────────────────────────────────────────────┘

Key Insight: The LLM translates natural language to structured form. QWED then verifies the structured form using deterministic engines. The LLM can be wrong — QWED catches and corrects errors.

​

Supported LLM providers

Provider	Env variable	Model variable	qwed init support
NVIDIA NIM	CUSTOM_API_KEY + CUSTOM_BASE_URL	CUSTOM_MODEL	Yes
OpenAI	OPENAI_API_KEY	OPENAI_MODEL	Yes
Anthropic	ANTHROPIC_API_KEY	ANTHROPIC_MODEL	Yes
Google Gemini	GOOGLE_API_KEY / GEMINI_API_KEY	GEMINI_MODEL	Yes
Ollama (local)	OLLAMA_BASE_URL	OLLAMA_MODEL	Manual
OpenAI-compatible	CUSTOM_BASE_URL + CUSTOM_API_KEY	CUSTOM_MODEL	Yes
Azure OpenAI	AZURE_OPENAI_ENDPOINT + AZURE_OPENAI_API_KEY	AZURE_OPENAI_DEPLOYMENT	Manual
Claude Opus	CLAUDE_OPUS_API_KEY	CLAUDE_OPUS_DEPLOYMENT	Manual
Custom (YAML)	Defined per provider	CUSTOM_MODEL	Manual (choose Custom Provider and enter the endpoint and model)

From v9.0.0, QWED names no default model. Set the model variable of the provider you use. QWED sends the value to the provider unchanged, so any model or deployment ID the provider serves works. When no model is set, the provider raises an error that names the variable to set. Nothing falls back to another model. On the server, ANTHROPIC_DEPLOYMENT still wins over ANTHROPIC_MODEL when it is set.

You can add any OpenAI-compatible provider (Groq, Together, Fireworks, and others) using a YAML configuration file. See Custom providers for details.

---

​

Configuration options

​

Option 1: use QWED’s built-in translation (recommended)

QWED can handle LLM translation internally:

from qwed import QWEDClient

# QWED uses its own LLM for translation
client = QWEDClient(api_key="qwed_your_key")

result = client.verify("What is 15% of 200?")
# QWED internally: "15% of 200" → 0.15 * 200 → verify with SymPy → 30

​

Option 2: bring your own LLM

Use QWED purely as a verification layer:

from qwed import QWEDClient
import openai

# Your LLM call
openai_client = openai.OpenAI(api_key="sk-...")
response = openai_client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is 847 × 23?"}]
)
llm_answer = response.choices[0].message.content

# QWED verification only (no LLM needed)
qwed = QWEDClient(api_key="qwed_your_key")
result = qwed.verify_math(
    expression="847 * 23",
    expected_result=llm_answer
)

if result.verified:
    print(f"✅ LLM was correct: {llm_answer}")
else:
    print(f"❌ LLM was wrong. Correct: {result.corrected}")

​

Option 3: Self-hosted with custom LLM

For self-hosted deployments, run qwed init or manually create a .env file:

# .env file — generated by `qwed init` or created manually
# WARNING: Contains secrets. NEVER commit this file.

# Active provider (openai, openai_direct, openai_compat, azure_openai,
# anthropic, claude_opus, gemini, ollama, auto)
ACTIVE_PROVIDER=openai

# Set the model variable for your provider. QWED names no default model.

# OpenAI
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o-mini

# OR Anthropic
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_MODEL=claude-sonnet-4-20250514

# OR Ollama (no key needed)
# OLLAMA_BASE_URL=http://localhost:11434/v1
# OLLAMA_MODEL=llama3

# OR Google Gemini
# GOOGLE_API_KEY=your-google-api-key
# GEMINI_MODEL=your-gemini-model

# OR OpenAI-compatible endpoint (Groq, Together, LM Studio, etc.)
# CUSTOM_BASE_URL=https://inference.do-ai.run/v1
# CUSTOM_API_KEY=your-key
# CUSTOM_MODEL=gpt-4o-mini

When ACTIVE_PROVIDER is not set, QWED defaults to Ollama as a safe local fallback. Set ACTIVE_PROVIDER explicitly if you want to use a cloud provider.

An ACTIVE_PROVIDER that names no provider QWED knows stops the request with an error naming ACTIVE_PROVIDER. A request that names its own provider must name one QWED knows, or it is refused. Each request runs on the provider it names, else on the one ACTIVE_PROVIDER selects.

QWED loads .env files in a specific order: the project-level .env (in the current working directory) takes precedence, followed by the global ~/.qwed/.env. This ensures your project-specific configuration always overrides global defaults. If python-dotenv is not installed, .env loading is skipped gracefully with a warning.

---

​

Provider-specific setup

You can configure any of these providers interactively by running qwed init.

​

OpenAI

export ACTIVE_PROVIDER=openai
export OPENAI_API_KEY=sk-your-key-here
export OPENAI_MODEL=gpt-4o-mini

Key format: sk-... or sk-proj-... — get yours at platform.openai.com/api-keys.

​

Anthropic (Claude)

export ACTIVE_PROVIDER=anthropic
export ANTHROPIC_API_KEY=sk-ant-your-key-here
export ANTHROPIC_MODEL=your-claude-model

Key format: sk-ant-... — get yours at console.anthropic.com/settings/keys.

The server’s Anthropic provider is an Azure AI Foundry client: set ANTHROPIC_ENDPOINT to your Foundry endpoint in the server’s environment. On the server, ANTHROPIC_ENDPOINT and ANTHROPIC_API_KEY are both required, or the provider stops with Missing Anthropic environment variables. The qwed CLI calls the Anthropic API directly instead: it reads ANTHROPIC_API_KEY and ANTHROPIC_MODEL, and stops if ANTHROPIC_ENDPOINT is set.

The Anthropic provider doesn’t send temperature, top_p, or top_k. The anthropic SDK has had no sampling parameters since 1.0, and Anthropic API version 2026-10-21, which the 2.x SDK sends, refuses them. Claude samples with the model’s defaults, and no QWED setting overrides them. Sampling can vary the translations Claude returns and the verdicts verify_fact and verify_image return. QWED’s fact and image verifiers record those verdicts only as advisory checks, never as VERIFIED.

​

Local LLMs (Ollama)

No API key needed. Install Ollama, pull a model, and configure QWED:

# Install and start Ollama
ollama serve

# Pull a model
ollama pull llama3

# Configure QWED
export ACTIVE_PROVIDER=ollama
export OLLAMA_BASE_URL=http://localhost:11434/v1
export OLLAMA_MODEL=llama3

​

OpenAI-compatible endpoint

For any service with an OpenAI-compatible API — DigitalOcean, Groq, Together AI, LM Studio, vLLM, and others:

export ACTIVE_PROVIDER=openai_compat
export CUSTOM_BASE_URL=https://inference.do-ai.run/v1
export CUSTOM_API_KEY=your-api-key
export CUSTOM_MODEL=gpt-4o-mini

CUSTOM_BASE_URL and CUSTOM_MODEL are required. If your endpoint does not require authentication (for example, a local vLLM or LM Studio server), you can omit CUSTOM_API_KEY entirely. QWED supplies a placeholder token so the underlying client initializes correctly.

​

Google Gemini

Gemini uses a native GeminiProvider powered by the google-generativeai SDK. Install the dependency first:

pip install google-generativeai

Then configure your environment:

export ACTIVE_PROVIDER=gemini
export GOOGLE_API_KEY=your-google-api-key
export GEMINI_MODEL=your-gemini-model

You can also use GEMINI_API_KEY instead of GOOGLE_API_KEY. Get yours at aistudio.google.com/app/apikey.

The Gemini provider supports math translation, logic verification, stats query generation, fact verification, and image claim verification. All API calls use a 30-second timeout and temperature=0.0 for deterministic output. If google-generativeai is not installed, QWED returns a structured ImportError instead of crashing.

​

Azure OpenAI

export ACTIVE_PROVIDER=azure_openai
export AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
export AZURE_OPENAI_API_KEY=your-azure-key
export AZURE_OPENAI_DEPLOYMENT=your-deployment-name
export AZURE_OPENAI_API_VERSION=2024-02-15-preview

​

Claude Opus

export ACTIVE_PROVIDER=claude_opus
export CLAUDE_OPUS_ENDPOINT=https://your-foundry-endpoint
export CLAUDE_OPUS_API_KEY=sk-ant-your-key
export CLAUDE_OPUS_DEPLOYMENT=claude-opus-4-5

Like the Anthropic provider, the Claude Opus provider doesn’t send temperature and samples with the model’s defaults. Sampling can vary translations and the verdicts verify_fact and verify_image return. QWED records those verdicts only as advisory checks, never as VERIFIED.

---

​

Universal provider config (YAML)

New in v4.0.0

You can define custom LLM providers using a YAML configuration file at ~/.qwed/providers.yaml. This is useful for managing multiple providers, custom endpoints, or community-contributed provider configs.

​

YAML format

providers:
  my-custom-llm:
    base_url: "https://api.example.com/v1"
    api_key_env: "MY_CUSTOM_API_KEY"
    default_model: "my-model-v2"
    models_endpoint: "/models"
    auth_header: "Authorization"
    auth_prefix: "Bearer"

Field	Required	Default	Description
base_url	Yes	—	The provider’s API base URL
api_key_env	Yes	—	Environment variable name containing the API key
default_model	No	—	Suggested model, recorded with the provider. The provider runs CUSTOM_MODEL, so set that variable (or enter the model in the wizard)
models_endpoint	No	/models	Endpoint for listing available models
auth_header	No	Authorization	HTTP header name for authentication
auth_prefix	No	Bearer	Prefix for the auth header value

The YAML file is written with 0600 permissions (owner-only) for security. The ~/.qwed/ directory is created with 0700 permissions.

​

Import community providers

You can import provider configurations from a URL:

from qwed_sdk.cli import import_provider

# Import a community-contributed provider config
import_provider("https://example.com/my-provider.yaml")

Or use the CLI:

qwed provider import https://example.com/my-provider.yaml

Imported providers are validated and sandboxed — only the allowed fields listed above are saved. Provider slugs are sanitized and cannot shadow built-in providers.

​

Key validation

QWED validates API keys in two stages:

1. Format check — Regex-based pattern matching (no network call). Built-in patterns include sk-... for OpenAI and sk-ant-... for Anthropic.
2. Connection test — Lightweight read-only request to the provider’s models endpoint to confirm the key works.

Run qwed init to validate your keys interactively, or test programmatically:

from qwed_sdk.cli import test_provider_connection

success, message = test_provider_connection("openai")

---

​

Provider routing

QWED automatically routes queries to the appropriate provider based on your configuration and query content.

​

Alias normalization

Provider names are normalized before routing, so common variations like openai-compatible, openai_compatible, and openai_compat all resolve to the same OpenAI-compatible provider. You do not need to worry about exact casing or separators when specifying a provider in API requests or environment variables.

​

Content-aware routing

When no preferred provider is specified in a request, QWED uses the configured default. For certain query types, QWED applies content-aware heuristics:

Query type	Routed to
Math/logic keywords (calculate, solve, equation, proof)	Your configured default provider
Creative/writing keywords (write, compose, essay, story)	Anthropic (Claude)
All other queries	Your configured default provider

You can always override routing by passing an explicit provider parameter in your API request.

---

​

Programmatic configuration

from qwed import QWEDClient, LLMConfig

# Option 1: Use environment variables (recommended)
client = QWEDClient()

# Option 2: Explicit configuration
client = QWEDClient(
    api_key="qwed_your_key",
    llm_config=LLMConfig(
        provider="openai",
        api_key="sk-...",
        model="gpt-4o",
        temperature=0.0,
    )
)

# Option 3: No LLM (verification only)
client = QWEDClient(
    api_key="qwed_your_key",
    llm_enabled=False  # Only use deterministic verification
)

---

​

Translation vs verification

Understanding the two phases:

Phase	What Happens	Who Does It	Required?
Translation	Natural language → Structured form	LLM (any)	Optional
Verification	Structured form → Proof	QWED Engines	Required

​

When you need an LLM

• client.verify("Is the derivative of x² equal to 2x?") — Needs LLM to parse
• client.verify("Calculate compound interest on $1000 at 5%") — Needs LLM

​

When you don’t need an LLM

• client.verify_math("diff(x**2, x) == 2*x") — Already structured
• client.verify_logic("(AND (GT x 5) (LT x 10))") — Already in DSL
• client.verify_sql("SELECT * FROM users") — Already structured
• client.verify_code("import os; os.system('rm -rf /')") — Code, not NL

---

​

FAQ

​

Do I need an LLM to use QWED?

No. If you’re sending structured queries (math expressions, SQL, code, QWED-Logic DSL), you don’t need an LLM. QWED engines work directly on structured input.

​

Can I use my own LLM and just use QWED for verification?

Yes. This is the “Bring Your Own LLM” pattern. Call your LLM, then pass its output to QWED for verification.

​

Which LLM is best for QWED translation?

QWED is model-agnostic and recommends no model. Use any model your provider serves: the deterministic engines check every translation it produces.

​

Is the LLM translation deterministic?

No. LLMs are inherently probabilistic. The Anthropic and Claude Opus providers don’t send temperature, so Claude samples with the model’s defaults. Sampling can vary translations and the verdicts verify_fact and verify_image return. QWED’s fact and image verifiers record those verdicts only as advisory checks, never as VERIFIED. The OpenAI, Azure OpenAI, OpenAI-compatible, Gemini, and Ollama providers send temperature=0.0, which reduces variation but doesn’t remove it.

That’s why QWED verification is essential. The deterministic engines give the same verdict for the same translation. They check the expression the model produced, not whether it matches your question, so two different translations of one question can get different verdicts.

---

​

Next steps

• Quick start tutorial
• Custom providers — add any OpenAI-compatible endpoint via YAML
• QWED-Logic DSL reference
• Self-hosting guide