Configuration

Complete reference example

listen: ":4000"
master_key: ${INFERIX_MASTER_KEY}

models:
  - name: owned/general-llm
    provider: owned
    endpoint: http://general-llm:8080/v1
    timeout_ms: 60000
    max_retries: 1

  - name: owned/slm-support
    provider: owned
    endpoint: http://slm-support:8080/v1
    timeout_ms: 15000
    labels:
      tier: cheap
      domain: cx

  - name: owned/slm-apiheal
    provider: owned
    endpoint: http://slm-apiheal:8080/v1
    timeout_ms: 10000
    labels:
      tier: cheap
      domain: apiheal

  - name: openai/gpt-4o-mini
    provider: openai
    api_key_env: OPENAI_API_KEY
    timeout_ms: 60000

routing:
  default: owned/general-llm
  fallback: openai/gpt-4o-mini
  rules:
    - name: support-cheap
      when:
        intent: support
      model: owned/slm-support
      fallback: owned/general-llm
    - name: apiheal-classify
      when:
        agent: api-healer
      model: owned/slm-apiheal
    - name: tenant-premium
      when:
        tenant: acme-premium
      model: owned/general-llm
      fallback: openai/gpt-4o-mini

observe:
  lensai: true
  traces: true
  sample_rate: 1.0
  attributes:
    - agent
    - tenant
    - intent
  otlp_endpoint: ${INFERIX_OTLP_ENDPOINT}

drift:
  enabled: true
  window: 24h
  golden_set: /var/lib/inferix/golden/support.jsonl
  min_samples: 50
  alert_threshold: 0.08
  notify:
    webhook: https://hooks.example.com/inferix-drift

fineforge:
  enabled: true
  teacher: owned/general-llm
  student: owned/slm-support
  auto_promote: false
  artifact_dir: /var/lib/inferix/fineforge

Top-level keys

models[]

• name — canonical id used in API model and RouteIQ (e.g. owned/general-llm).
• provider — owned | openai | anthropic | custom.
• endpoint — base URL for owned/custom servers.
• api_key_env — env var name holding the provider key.
• timeout_ms, max_retries, labels — operational metadata for LensAI filters.

routing

Rules are evaluated top-down; first match wins. Unmatched calls use default, then fallback on invoke failure. Match keys: intent, agent, tenant, and request attributes. Full policy guide: Routing policies.

observe

• lensai: true — emit metrics per call.
• traces: true — emit TraceForge spans.
• sample_rate — 0.0–1.0 for span sampling (metrics stay full).
• otlp_endpoint — optional export to your collector.

drift & fineforge

window and alert_threshold drive DriftWatch. auto_promote: false is recommended until you trust the golden set. See Drift & retrain.