ADR-0005 — Apex Model Family Replaces Anthropic Proxy

Field	Value
Status	🟢 Accepted (rollout pending)
Date	2026-04
Deciders	@aphaiboon
Related ADRs	(none — first model-family decision)
Related incidents	FM-018 — AnthropicProvider context_management to Haiku — motivating example of provider-specific quirks

Context

CMMD’s Sidekick currently routes through an Anthropic proxy (Haiku for FAST tier, Sonnet for CAPABLE, Opus for FLAGSHIP — see Sidekick Routing Pipeline). This works well today but has three structural problems:

1. Cost. At scale, Anthropic API spend dominates COGS. Every Sidekick chat hits the proxy. Cost-per-active-user is approximately 4–6× infrastructure cost.
2. Privacy positioning. Anthropic explicitly does not train on API data, but customers (especially enterprise) still ask “does my data leave your servers?” The answer today is “yes, to Anthropic’s servers.” Acceptable for SMB but a real objection at enterprise.
3. Product positioning. The Sidekick is the product. Branding our own models — “CMMD 1.0 Fast / CMMD 1.0 / CMMD 1.0 Pro” — strengthens the positioning that the AI is ours, not a generic copilot wrapped around a third-party model.

We’ve been training and evaluating an open-source-grounded model family called Apex. Internal benchmarks are within striking distance of Anthropic Sonnet for our specific workloads (tool-calling, context synthesis, code).

Decision

Replace the Anthropic proxy with the Apex model family across all Sidekick tiers, gated by feature flag for staged rollout.

Model mapping

Apex	Replaces	Base model	Tier
Apex Flash	FAST_MODEL (Haiku)	Qwen 3.5 9B	trivial + simple
Apex General	CAPABLE_MODEL (Sonnet)	Qwen 3.5 122B-A10B	standard
Apex Reason	FLAGSHIP_MODEL (Opus)	DeepSeek-R1 32B	complex
Apex Code	(new — code-specialist tier)	Qwen3-Coder-Next 80B-A3B	code mode
Apex Vision	(new — vision/image)	Chroma1	image understanding/generation

→ Full Model Card with capabilities, limitations, safety: Model Card — Apex Family

Customer-facing display names

Internal constant	Customer-facing name
FAST_MODEL	CMMD 1.0 Fast
CAPABLE_MODEL	CMMD 1.0
FLAGSHIP_MODEL	CMMD 1.0 Pro

Never hardcode raw model strings — always route through AIGateway using these constants. CLAUDE.md rule.

Alternatives considered

Option A — Stay on Anthropic indefinitely

• ✅ Best-in-class quality, ship now
• ✅ Zero ops burden (no GPU infra)
• ❌ Cost trajectory unsustainable at scale
• ❌ Privacy/positioning weaker
• Rejected: Strategic constraint, not technical.

Option B — Multi-provider (Anthropic + OpenAI + open-source via OpenRouter)

• ✅ Hedges single-vendor risk
• ✅ Cost optimization per tier
• ❌ Operational complexity
• ❌ Inconsistent behavior across providers makes prompt eng harder (see FM-018)
• ❌ Doesn’t solve the strategic “the AI is ours” positioning
• Rejected: Helps cost, doesn’t help strategy. We already use OpenRouter for non-customer-facing background jobs (qwen3-30b-a3b).

Option C — Build a tiny custom model from scratch

• ❌ 2–3 years of work
• ❌ Quality risk too high

Option D — License a private model (Cohere, Mistral, etc.)

• ✅ Decent quality, less ops burden than hosting our own
• ❌ Still a third party — privacy positioning unchanged
• ❌ Per-token cost similar to Anthropic
• Rejected: Doesn’t solve the strategic concern.

Consequences

Positive

• ✅ COGS reduction (40–60% estimated based on internal load)
• ✅ Customer data never leaves CMMD infrastructure
• ✅ Branded model family — “CMMD 1.0” SKUs
• ✅ Custom fine-tuning possible (e.g. tool-calling specialization)
• ✅ No third-party rate limits constraining product growth

Negative / trade-offs

• ⚠️ Self-hosted GPU infrastructure to operate and maintain
• ⚠️ Quality below Anthropic Opus on the longest-tail complex queries (acceptable for v1; planned for v2)
• ⚠️ Eval discipline becomes critical — we can’t lean on Anthropic’s safety + quality work
• ⚠️ Initial CapEx for GPU infra
• ⚠️ Customer onboarding may not notice the change (transparent transition) — risk of “this seems worse” if quality regression slips through eval

Risks

• 🔴 Quality regression risk — the most important risk. Mitigation: A/B test against Anthropic on a 5% traffic slice for 4 weeks before broad rollout. Feature flag default off; manual ramp.
• 🔴 Safety/jailbreak resistance — Anthropic invested years in Claude’s safety posture. Apex needs its own red-team pass. Mitigation: dedicated safety eval suite, external red-team engagement before GA.
• 🟡 Ops burden — running models 24/7 needs on-call. Mitigation: on-call runbook + alerting on inference latency + error rate.
• 🟡 Cold-start latency for serverless GPU. Mitigation: keep-warm strategy or provisioned capacity for production.

Implementation plan

• Train Apex Flash, General, Reason
• Train Apex Code (Qwen3-Coder-Next base) — in progress
• Train Apex Vision (Chroma1 base) — in progress
• Build inference infrastructure — dedicated GPU cluster + serverless fallback
• Build the AIGateway abstraction so swap is a 3-file change (model-constants.ts, model-display-names.ts, ai-model-registry-service.ts)
• Eval suite covering tool-calling, context synthesis, code, refusals
• A/B test: 5% traffic on Apex, 95% on Anthropic for 4 weeks
• Safety red-team engagement before GA
• Customer comms: “Introducing CMMD 1.0” announcement
• Cut over by tier: Flash (lowest risk) first, then General, then Reason

Success metrics

Metric	Baseline (Anthropic)	Target (Apex)
User-perceived quality (Likert)	4.3/5	≥4.2/5
Tool-call accuracy	91%	≥89%
P50 latency (CAPABLE tier)	6s	≤7s
Cost per active user / month	$X	≤$0.4X
Refusal rate on benign queries	<2%	<3%

Review

Next review: 4 weeks after GA, then quarterly. Reassess if any success metric drops below threshold for 2 consecutive weeks.

References

• Model Card — Apex Family — full per-model spec + safety
• Sidekick Routing Pipeline
• App — Sidekick
• Security Posture + Privacy & Data Handling
• FM-018 — AnthropicProvider context_management to Haiku
• Code: server/services/router/model-constants.ts, complexity-evaluator.ts, ai-gateway.ts