ADR-0002 — Apex Harness Uses Native Anthropic-Compat Endpoints

Status: Active — implemented 2026-05-19.

Decision

Both CMMD and Forge call a local LiteLLM proxy speaking the native Anthropic Messages API (/v1/messages) — same SDK code as cloud Claude, just different baseURL. Two parallel Cloudflare tunnels expose the proxies to a stable hostname (apex-api.cmmd.ai, lab-apex-api.cmmd.ai). Master key authentication via x-api-key header.

Context

CMMD currently calls claude-{haiku,sonnet,opus} directly via @anthropic-ai/sdk. Forge uses the Claude Agent SDK with per-profile baseUrl overrides. The team needs a local LLM serving solution that:

• Works on Mac Studio (M1 Ultra 128GB) and Ryzen (RTX 2080 Super)
• Doesn’t require changing SDK clients in CMMD or Forge
• Supports vision input, tool calling, streaming
• Provides Anthropic-format responses (so CMMD’s content-block handling works unchanged)
• Stays under near-$0 cost during alpha

Decision detail

Architecture

Two endpoints on the same wire format, different hosts:

Endpoint	Hosts	Tier
apex-api.cmmd.ai	Mac Studio (Sentinel)	apex-* production
lab-apex-api.cmmd.ai	Ryzen	lab-* experimental

Both expose Anthropic Messages API (/v1/messages) and OpenAI Chat Completions API (/v1/chat/completions) via LiteLLM. Shared master key authenticates against both.

Why parallel tunnels (not one proxy fronting both)

Sentinel cannot reach Ryzen over the user’s tailnet because it’s “shared in” from cmmd.ai. We considered:

• Installing a second Tailscale daemon on Sentinel (blocked: requires cmmd.ai tailnet admin)
• Having Sentinel proxy to Ryzen via Cloudflare loop (works but adds 50-100ms per ryzen request)
• Parallel tunnels (chosen): symmetric, low-latency, independent

Apps just configure two base URLs. The aliases (apex-core, lab-flash, lab-general) tell them which endpoint to use.

Why local model serving via LiteLLM (not direct Ollama)

• Ollama speaks OpenAI Chat Completions format only. CMMD/Forge speak Anthropic format.
• LiteLLM proxies translate between formats. The Anthropic-format endpoint preserves tool_use/tool_result blocks, vision base64, streaming SSE shape — what CMMD’s AnthropicProvider.ts already handles.
• Adds ~5ms overhead per request. Acceptable.

Alternatives rejected

Alternative	Rejected because
Pure cloud (Claude / DeepSeek / Kimi)	Real per-token cost, no privacy guarantee, no fallback if cloud rate-limits
Local Ollama OpenAI-compat direct	CMMD/Forge code would need OpenAI translation everywhere
Second Tailscale daemon on Sentinel	Requires tailnet admin access we don’t have
Single LiteLLM on Sentinel proxying to Ryzen via Cloudflare loop	Extra 50-100ms latency on every lab-tier request

Implementation

Deployed 2026-05-19:

• Mac Studio: LaunchAgents for Ollama, LiteLLM, cloudflared, node_exporter
• Ryzen: systemd units for LiteLLM, cloudflared-lab-apex (Ollama already running)
• Cloudflare: 2 tunnels (apex, lab-apex) + 2 DNS CNAMEs in cmmd.ai zone (Zswire@gmail.com account)
• Master key generated, stored in macOS Keychain on Alex’s laptop + both host machines

Source of truth: cmmd-center/apex.

Consequences

Wins:

• ~$0/month per-token cost for the bulk of inference
• Privacy (queries don’t leave our infra)
• Stable URLs (apps don’t break when we swap models)
• Cloud fallback (cortex-* aliases via LiteLLM if needed)

Tradeoffs:

• Two endpoints instead of one — apps configure two base URLs (acceptable)
• Mac Studio is single-point-of-failure for apex-core (mitigated by cloud fallback)
• No cache_control (Claude-only feature) — minor cost impact at scale
• Tailscale ACL asymmetry forced parallel tunnels

Related

• ADR-0003: Apex Model Choices — sister decision on which models to deploy
• Apex Architecture
• cmmd-center/apex