Why cloud LLMs for QWED verification?

TL;DR: On the paths that use a model, the model translates your query into a formal statement and a deterministic engine evaluates it. Because the translation is not proven, the trust-boundary step can still return INCONCLUSIVE even when that evaluation succeeds (see the Math engine). A model that translates more reliably gives fewer failed verifications. Larger hosted models (GPT-4 class, Claude) usually translate more reliably than small local models, but measure this on your own workload: QWED publishes no accuracy figures for any model.

🎯 The core problem

Verification requires two things:

LLM translates the query (natural language → structured reasoning)
Symbolic verifier proves the answer (SymPy, Z3, AST)

If the LLM gets step 1 wrong, verification fails even with perfect symbolic math. The translation itself is not verified: every result includes the formal statement, so a person can check that it says what was meant.

📊 How model answer quality shows up

Math verification example

Query: “What is the integral of 2x?” For this query, a correct model answer is “x² + C”. A smaller model may instead answer “2x²” or confuse the derivative with the integral. QWED does not publish accuracy rates per model; try your own queries against the models you are considering.

Why this matters

When QWED verifies:

LLM says: "x² + C"
SymPy computes: integrate(2*x, x) = x**2
QWED compares: ✅ MATCH!

If LLM is wrong:

LLM says: "2x²" (incorrect)
SymPy computes: x**2
QWED: ❌ NO MATCH → Verification fails

Result: User sees failure, even though QWED’s symbolic engine is correct!

🤔 When to use each

Use Case	Local LLM (Ollama)	Cloud LLM (OpenAI/Anthropic)
Development/Testing	✅ Free, fast iteration	⚠️ Costs add up
Production (Critical)	⚠️ Test translation quality first	✅ Recommended
Privacy-Sensitive Data	✅ 100% local + PII masking	⚠️ Use with PII masking
Cost-Sensitive	✅ $0/month	⚠️ ~$5-50/month
High-Stakes Decisions	⚠️ More failed verifications likely	✅ Recommended

💡 QWED’s hybrid approach

Best Practice: Use both strategically

Development setup (free)

from qwed_sdk import QWEDLocal

# Local LLM for development
client_dev = QWEDLocal(
    base_url="http://localhost:11434/v1",  # Ollama
    model="llama3",
    cache=True  # Cache responses
)

# Test your queries
result = client_dev.verify("What is 2+2?")

Cost: $0/month
Use for: Prototyping, experimentation, learning

Production setup (reliable)

import os
from qwed_sdk import QWEDLocal

# Cloud LLM for production
client_prod = QWEDLocal(
    provider="openai",
    api_key=os.getenv("OPENAI_API_KEY"),
    model="gpt-4o-mini",
    mask_pii=True,   # Privacy protection
    cache=True        # 50-80% cost savings!
)

# Critical verification
result = client_prod.verify("Verify calculation: ...")

Cost: ~$5-10/month (with caching!)
Use for: Production, high-stakes decisions

💰 Cost analysis

Local LLM (Ollama)

Setup: 10 minutes (download model)
Monthly Cost: $0
Translation quality: depends on model size; measure on your workload
Privacy: 100% local
Best for: Development, testing, learning

Cloud LLM (OpenAI GPT-4o-mini)

Setup: 2 minutes (API key)
Monthly Cost: $5-10 (with caching)
Translation quality: usually higher than small local models; measure on your workload
Privacy: Use PII masking
Best for: Production, critical tasks

With QWED caching (cost savings)

# First query: Hits LLM (costs $$)
result1 = client.verify("What is 2+2?")

# Same query within 24 hours: Cache hit (FREE!)
result2 = client.verify("What is 2+2?")  # $0 cost!

Savings: repeated queries hit the cache and skip the LLM call.

🔒 Privacy considerations

Local LLM advantages

✅ Private — data stays on your machine
✅ No API keys - no third-party access
✅ Data residency - no third party processes the data (this alone does not make a system GDPR or HIPAA compliant)

Cloud LLM with PII masking

client = QWEDLocal(
    provider="openai",
    model="gpt-4o-mini",
    mask_pii=True,  # Auto-mask emails, SSNs, etc.
    pii_entities=["EMAIL_ADDRESS", "CREDIT_CARD", "US_SSN"]
)

# Sensitive data protected!
result = client.verify("User email: john@example.com, calculate 2+2")
# OpenAI sees: "User email: <EMAIL_ADDRESS>, calculate 2+2"

Result: A hosted model sees masked text instead of the raw values. Masking is pattern-based, so check what your entities list covers. 🔒

🎯 Recommendation by use case

Healthcare (HIPAA)

# Option 1: Local LLM (most private)
client = QWEDLocal(
    base_url="http://localhost:11434/v1",
    model="llama3"
)

# Option 2: Cloud + PII masking (more accurate)
client = QWEDLocal(
    provider="openai",
    model="gpt-4o-mini",
    mask_pii=True,
    pii_entities=["PERSON", "US_SSN", "MEDICAL_LICENSE"]
)

Recommendation: Cloud + PII masking for critical diagnoses

Finance (PCI-DSS)

client = QWEDLocal(
    provider="openai",
    model="gpt-4o-mini",
    mask_pii=True,
    pii_entities=["CREDIT_CARD", "IBAN_CODE"]
)

Recommendation: Cloud + PII masking (accuracy matters for money!)

Enterprise (general)

# Development
dev_client = QWEDLocal(base_url="http://localhost:11434/v1", model="llama3")

# Production
prod_client = QWEDLocal(provider="openai", model="gpt-4o-mini", mask_pii=True, cache=True)

Recommendation: Hybrid approach

🚀 The QWED advantage

With a local LLM, the deterministic check still runs.

Scenario: local LLM makes mistake

client = QWEDLocal(base_url="http://localhost:11434/v1", model="llama3")

# Llama 3 might say: "Derivative of x² is x" (WRONG!)
result = client.verify("What is the derivative of x²?")

# QWED's symbolic verification:
# SymPy: diff(x**2, x) = 2*x
# LLM said: "x"
# QWED: ❌ NO MATCH! 
# result.verified = False

User sees: “Verification failed - LLM answer doesn’t match symbolic proof” But:

More failures = worse UX
Cloud LLMs = fewer verification failures = better UX

📈 Measuring it yourself

QWED does not publish per-model accuracy figures. To choose a model, run a sample of your real queries through each candidate and compare how often the formal statement in the result matches what you meant, and how often verification fails because of a bad translation. QWED’s own evaluation results are at qwedai.tech/evidence.

🎓 Bottom line

Start with local LLM

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Download model
ollama pull llama3

# Use with QWED
python -c "from qwed_sdk import QWEDLocal; \
  client = QWEDLocal(base_url='http://localhost:11434/v1', model='llama3'); \
  print(client.verify('2+2'))"

Perfect for: Learning, prototyping, hobby projects

Scale to cloud LLM

# Get API key from OpenAI
export OPENAI_API_KEY="sk-..."

# Use with QWED
python -c "from qwed_sdk import QWEDLocal; \
  client = QWEDLocal(provider='openai', model='gpt-4o-mini', mask_pii=True, cache=True); \
  print(client.verify('2+2'))"

Perfect for: Production, enterprise, critical decisions

LLM configuration guide — complete LLM setup
PII masking guide — privacy protection
Caching guide — cost savings

❓ FAQ

Q: Can I use Llama 3 70B instead of GPT-4?
A: Yes. Larger local models (70B+) usually translate more reliably than small ones but need significant hardware (40GB+ VRAM). Q: Is Ollama really free?
A: Yes! Fully open source. You just need hardware to run it. Q: What about Google Gemini?
A: QWED supports Gemini. Test its translation quality on your own queries, as with any model. Q: Can I switch between local and cloud?
A: Absolutely! Change the provider parameter anytime. Q: Do I need PII masking with local LLMs?
A: Not necessarily, but it’s still good practice for audit trails.

The choice is yours - QWED works with both! 🚀 Recommendation: Start local (free), scale to cloud (reliable) when it matters.

Architecture

Concepts

Specifications

Integration Patterns

Contributing

Why cloud LLMs for QWED verification?

🎯 The core problem

📊 How model answer quality shows up

Math verification example

Why this matters

🤔 When to use each

💡 QWED’s hybrid approach

Development setup (free)

Production setup (reliable)

💰 Cost analysis

Local LLM (Ollama)

Cloud LLM (OpenAI GPT-4o-mini)

With QWED caching (cost savings)

🔒 Privacy considerations

Local LLM advantages

Cloud LLM with PII masking

🎯 Recommendation by use case

Healthcare (HIPAA)

Finance (PCI-DSS)

Enterprise (general)

🚀 The QWED advantage

Scenario: local LLM makes mistake

📈 Measuring it yourself

🎓 Bottom line

Start with local LLM

Scale to cloud LLM

❓ FAQ

​🎯 The core problem

​📊 How model answer quality shows up

​Math verification example

​Why this matters

​🤔 When to use each

​💡 QWED’s hybrid approach

​Development setup (free)

​Production setup (reliable)

​💰 Cost analysis

​Local LLM (Ollama)

​Cloud LLM (OpenAI GPT-4o-mini)

​With QWED caching (cost savings)

​🔒 Privacy considerations

​Local LLM advantages

​Cloud LLM with PII masking

​🎯 Recommendation by use case

​Healthcare (HIPAA)

​Finance (PCI-DSS)

​Enterprise (general)

​🚀 The QWED advantage

​Scenario: local LLM makes mistake

​📈 Measuring it yourself

​🎓 Bottom line

​Start with local LLM

​Scale to cloud LLM

​🔗 Related documentation

​❓ FAQ

🎯 The core problem

📊 How model answer quality shows up

Math verification example

Why this matters

🤔 When to use each

💡 QWED’s hybrid approach

Development setup (free)

Production setup (reliable)

💰 Cost analysis

Local LLM (Ollama)

Cloud LLM (OpenAI GPT-4o-mini)

With QWED caching (cost savings)

🔒 Privacy considerations

Local LLM advantages

Cloud LLM with PII masking

🎯 Recommendation by use case

Healthcare (HIPAA)

Finance (PCI-DSS)

Enterprise (general)

🚀 The QWED advantage

Scenario: local LLM makes mistake

📈 Measuring it yourself

🎓 Bottom line

Start with local LLM

Scale to cloud LLM

🔗 Related documentation

❓ FAQ