Sidekick Routing Pipeline
How a user message becomes a response. Four tiers (trivial, simple, standard, complex) + trivial fast-path. Performance + correctness depend on it being right.
How a user message becomes a response. Every chat hits this pipeline. Performance + correctness depend on it being right.
The four tiers
Section titled “The four tiers”| Tier | Model | Tools | Context Gather | Typical Latency | When |
|---|---|---|---|---|---|
| trivial | FAST_MODEL | disabled | skipped | < 2s | Greetings, time/date, identity, chitchat, acks |
| simple | FAST_MODEL | enabled | minimal | 3–5s | Short factual lookups, single-entity queries |
| standard | CAPABLE_MODEL | enabled | medium | 5–15s | Synthesis, search, multi-step, external, actions |
| complex | FLAGSHIP_MODEL | enabled | deep | 15–30s | Reports, deep analysis, strategy |
Constants live in server/services/router/model-constants.ts — never hardcode raw provider model strings.
The trivial fast-path (the key optimization)
Section titled “The trivial fast-path (the key optimization)”Pure regex classification in IntentClassifier.TRIVIAL_PATTERNS that fires before pattern scoring and LLM classification. When matched:
- Skips
enrichWithTools - Skips
preGatherContext - Skips Brain fetch
- Skips live queries
- Skips thinking mode
- Goes straight to
FAST_MODELwhich answers from the system prompt (time, user name, timezone are always injected there)
Impact: “what time is it” went from ~30s → ~2s.
Safety guards
Section titled “Safety guards”Queries are never trivially fast-pathed if:
- They contain action verbs (
create,add,schedule, etc.) - They’re longer than 80 chars
- Search mode is not
smart
Adding patterns
Section titled “Adding patterns”- Add a regex to
TRIVIAL_PATTERNSinintent-classifier.ts - Add unit tests in
tests/unit/router/trivial-fast-path.test.ts - Run
npm run test:unit -- trivial - Update this page with the new pattern category
Mode auto-elevation
Section titled “Mode auto-elevation”User-selected modes:
| User picks | What that means |
|---|---|
| Quick | ”I just want an answer.” Tools disabled by default. |
| Autopilot (default) | “Do what you think is right.” Full tools, full context. |
| Deep Think | ”Take your time, reason harder.” Extended thinking, flagship model. |
| Code | ”I’m pairing on code.” Sandbox tools, 32k token budget. |
Auto-elevation: if user picks Quick but their message needs a tool, the Sidekick transparently elevates to Autopilot for that turn. Implemented in buildModeElevation() in sidekick-chat-runtime-service.ts.
Search scope
Section titled “Search scope”Users can scope Sidekick’s knowledge per chat:
| Scope | Behavior |
|---|---|
| Smart (default) | Full pipeline: Brain + web + memory |
| Internal | Brain + memory only — skip web |
| External | Web only — skip Brain/memory/live queries |
Slash commands /quick, /auto, /think switch modes mid-conversation.
Context envelope
Section titled “Context envelope”Every Sidekick turn assembles a context envelope before calling the model. Order of operations:
- Ambient context (always injected): explicit day name, full date, timezone, user, active integrations
- Context tags (explicit
@or+mentions): person, project, Brain doc, Brain folder- When explicit Brain tags are present, auto Brain semantic search is skipped — pinned context wins
- Brain document tag: injects up to 8k chars of content (or summary + tool hint for file-backed)
- Brain folder tag: injects up to 5 docs × 1500 chars from the folder
- Auto Brain semantic search (only if no explicit Brain tags): top results above similarity threshold
- Live queries: tasks due today, calendar events in next 4h, unread thread messages
- Memory: working memory snippets relevant to the message
- Conversation history: previous N turns in the same chat/thread
Resolved by context-envelope-loader.ts and the contextTagResolverRegistry.
The Apex migration path
Section titled “The Apex migration path”The current routing uses an Anthropic proxy. The constants will swap to CMMD’s Apex models with a 3-file change:
| File | Change |
|---|---|
model-constants.ts | FAST_MODEL = 'apex-flash', etc. |
model-display-names.ts | Update display strings (CMMD 1.0 Fast, etc.) |
ai-model-registry-service.ts | Register Apex providers |
No routing logic changes needed. Tier semantics stay identical. See ADR-0005 — Apex Model Family.
What can go wrong
Section titled “What can go wrong”| Failure | Symptom | Fix |
|---|---|---|
| Trivial fast-path matches an action verb | ”Create a task” gets a 2s placeholder response | Tighten regex, add safety guards |
| Tool call returns malformed JSON | Sidekick hangs or errors | Tool-handler Zod validation; surface failure to user |
| Mode auto-elevation loop | Quick → Autopilot → Quick endless flip | Elevation is one-shot per turn — guard at runtime layer |
| Brain semantic search times out | Sidekick blocks for 10+s | Timeout budget + fallback to no-context |
| Context envelope > token limit | Provider 4xx | Truncate envelope by priority: keep ambient + tags, drop older history |
Reference code
Section titled “Reference code”| Concern | File |
|---|---|
| Tier classification | server/services/router/intent-classifier.ts |
| Trivial patterns | Same file, TRIVIAL_PATTERNS constant |
| Model constants | server/services/router/model-constants.ts |
| Complexity evaluator | server/services/router/complexity-evaluator.ts |
| Chat runtime orchestration | server/services/sidekick/sidekick-chat-runtime-service.ts |
| Context envelope | server/services/sidekick/context-envelope-loader.ts |
| Mode instructions | server/services/sidekick/system-prompt-builder.ts |
| AI Gateway | server/services/ai/ai-gateway.ts |