Apex — Models
Per-alias capabilities, sizes, tool-use status, and Claude Code CLI compatibility — verified 2026-05-19.
Source of truth: This page reflects what’s actually on disk on Mac Studio and Ryzen as of 2026-05-19, verified via
ollama show,ollama list, and end-to-end smoke tests through LiteLLM. See Tool use verification for the full test results.
Tier conventions
Section titled “Tier conventions”| Prefix | Tier | Where it runs | Branding |
|---|---|---|---|
apex-* | Production premium | Mac Studio (M1 Ultra 128GB) | Client-facing |
lab-* | Experimental | Ryzen (RTX 2080 Super 8GB + 32GB RAM) | Internal only |
cortex-* | Cloud passthrough | (reserved) | Utility |
Deployed model aliases
Section titled “Deployed model aliases”apex-core
Section titled “apex-core”| Property | Value |
|---|---|
| Status | 🟢 Production-ready |
| Public endpoint | https://apex-api.cmmd.ai |
| Localhost endpoint | http://mac-studio:4000 (LAN only, no CF timeout) |
| Backing model | qwen3.5:122b-a10b |
| Ollama blob ID | 8b9d11d807c5 |
| Size on disk | 81 GB (Q4_K_M) |
| Resident VRAM (loaded) | 95 GB, 100% GPU |
| Architecture | qwen35moe — 125.1B total params / 10B active (MoE) |
| Context window | 262,144 tokens |
| Embedding dim | 3,072 |
| Ollama capabilities | completion, vision, tools, thinking |
| Renderer / Parser | qwen3.5 (built-in to Ollama 0.17+) |
| Tool use (curl /v1/messages) | ✅ VERIFIED — returns proper tool_use content blocks, stop_reason: tool_use |
| Claude Code CLI (localhost) | ✅ VERIFIED end-to-end — claude --print executed LS tool, returned correct result. ~8min/turn cold, faster warm. |
| Claude Code CLI (via Cloudflare) | ⚠️ Blocked by CF 524 (100s gateway timeout). LAN-direct only for now. |
| Deployed | 2026-05-19 (Mac Studio Ollama 0.17.7) |
| Use case | Primary production. Default for new CMMD/Forge integrations. Best Apex model for tool use. |
lab-flash
Section titled “lab-flash”| Property | Value |
|---|---|
| Status | 🟢 Production-ready (low-latency tier) |
| Public endpoint | https://lab-apex-api.cmmd.ai |
| Localhost endpoint | http://ryzen:4000 (LAN only) |
| Backing model | haervwe/GLM-4.6V-Flash-9B:latest |
| Ollama blob ID | ad2e2e374c6b |
| Size on disk | 8.0 GB (Q4_K_M) |
| Architecture | glm4 dense 9B (9.4B params) |
| Context window | 131,072 tokens (with num_ctx: 65556 Ollama default) |
| Embedding dim | 4,096 |
| Vision projector | CLIP, 892M params, 1536-dim embedding |
| Ollama capabilities | completion, vision, tools, thinking |
| Template style | Native Go template with <|system|> / <tool_call> markers |
| Tool use (curl /v1/messages) | ✅ VERIFIED — returns proper tool_use content blocks, stop_reason: tool_use |
| Claude Code CLI (localhost) | ✅ Expected to work (same surface as apex-core; not exercised end-to-end yet) |
| Claude Code CLI (via Cloudflare) | ⚠️ Likely same CF 524 issue with cold-load + large schemas |
| Deployed | 2026-05-19 (Ryzen Ollama 0.20.2) |
| Use case | Fast multimodal — fits Ryzen’s 8GB GPU entirely. Vision-driven function calling. Quick classification / routing / OCR-style tasks. |
lab-general
Section titled “lab-general”| Property | Value |
|---|---|
| Status | 🟡 Tool use BROKEN — chat/text only |
| Public endpoint | https://lab-apex-api.cmmd.ai |
| Localhost endpoint | http://ryzen:4000 (LAN only) |
| Backing model | huihui_ai/Qwen3.6-abliterated:27b |
| Ollama blob ID | 418838acbea7 |
| Size on disk | 17 GB (Q4_K_M) |
| Resident memory (loaded) | 23 GB total — 4.5 GiB on GPU + 11.7 GiB on CPU (only 21/65 layers on GPU) |
| Architecture | qwen35 dense 27.8B (despite “3.6” branding, GGUF reports qwen35 family; abliterated for uncensored output) |
| Context window | 262,144 tokens (with num_ctx: 4096 Ollama default at load) |
| Embedding dim | 5,120 |
| Ollama capabilities | completion, vision, tools, thinking (declared but tools broken in practice) |
| Tool use (curl /v1/messages) | ❌ BROKEN — emits tool call as raw text inside thinking block ({"name":"get_weather","parameters":...}), Ollama parser doesn’t extract it as a tool_use block. Returns stop_reason: max_tokens. Root cause: abliterated model uses OpenAI-style "parameters" instead of Qwen-native "arguments". |
| Latency | ❌ Unusable interactively — 4m24s for 256 output tokens because most weights run on CPU. Cloudflare 524 timeout fires before responses complete. |
| Claude Code CLI | ❌ Not viable (both reasons above) |
| Deployed | 2026-05-19 (Ryzen Ollama 0.20.2) |
| Use case | Text-only experimental workhorse. Suitable for batch doc drafting where tool-calling isn’t needed and slow response is acceptable. Do not route Claude Code CLI traffic here. |
How to pick
Section titled “How to pick”| Use case | Pick | Why |
|---|---|---|
| Claude Code CLI on developer laptop | apex-core via LAN-direct | Only Apex model verified end-to-end with Claude Code |
| CMMD sidekick chat | apex-core | Largest local model, strongest reasoning, tool-use verified |
| Forge IDE assistance | apex-core | Same |
| Quick classification / routing | lab-flash | Fastest, fits Ryzen GPU entirely, tools work |
| Vision-heavy automation (OCR, chart parsing) | lab-flash | Native vision-driven function calling |
| Internal doc drafting (no tool calls needed) | lab-general | Slow but capable for batch text work |
Anything requiring cache_control | Cloud (Claude, DeepSeek) — Apex doesn’t support | — |
| Anything tool-heavy with strict latency budget | Cloud Claude until Apex GPU scaled | Apex models are minutes/turn, not seconds |
Known limitations
Section titled “Known limitations”Cloudflare 524 gateway timeout (100 seconds)
Section titled “Cloudflare 524 gateway timeout (100 seconds)”The CF tunnels (apex-api.cmmd.ai, lab-apex-api.cmmd.ai) inherit a 100-second proxy read timeout on the Pro plan (Enterprise can extend to 600s). This is fatal for:
- Cold-start requests on
apex-core(~120s to load 95GB into VRAM) - Any
lab-generalrequest (4+ minutes due to GPU/CPU split) - Long Claude Code CLI turns where the model thinks for >100s before emitting first token
Mitigations:
- LAN-direct for Claude Code CLI: use
http://mac-studio:4000from the office network (no CF in path) - Pre-warm the model: run a short request to load the model into VRAM before the real workload (
ollama psshould show the model resident withUNTILin the future) - Streaming responses: when client supports streaming, CF lets the connection live as long as bytes are flowing — only 524s when there’s >100s of silence. Verify LiteLLM is configured to stream upstream from Ollama.
- Cloudflare plan upgrade: Enterprise tier extends timeout to 600s. ~$200/mo+. Probably not needed if streaming works.
lab-general tool-use parser mismatch
Section titled “lab-general tool-use parser mismatch”Abliterated Qwen3 variants from huihui_ai emit tool calls in OpenAI format ({"name": "...", "parameters": {...}}) instead of Qwen-native format (<tool_call>{"name":"...","arguments":{...}}</tool_call>). Ollama’s qwen3.5 parser expects the native format, so the call is dropped.
Fixes (untested):
- Use
huihui_ai/qwen3-abliterated:14binstead (smaller, may have correct template) - Override with a Modelfile that adds the
<tool_call>...</tool_call>wrapping in the template - Switch
lab-generalto a non-abliterated 27B Qwen variant
Ryzen GPU is too small for 27B
Section titled “Ryzen GPU is too small for 27B”8GB VRAM on the RTX 2080 Super fits only 21/65 layers of a 27B model at Q4. The remaining 44 layers run on CPU, dragging inference to 4+ min per 256 tokens. Either:
- Stick to ≤14B models on Ryzen (
qwen3-abliterated:14bwould fit fully) - Upgrade Ryzen GPU to 24GB+ (RTX 3090/4090) for 27B at speed
- Move the 27B class to a cloud burst instance
Adding a new model
Section titled “Adding a new model”- Pull on the appropriate host:
ssh <host> 'ollama pull <model>' - Verify capabilities:
ollama show <model>— confirmtoolsis in the Capabilities list - Smoke-test tool use: send a
/v1/messagescurl withtools: [...]and verify atool_useblock comes back (see tool-use-verification for the exact curl) - Add alias to the corresponding LiteLLM config (
apex/mac-studio/litellm/config.yamlorapex/ryzen/litellm/config.yaml) - Add to
apex/models/registry.ymlunder the right tier - Open PR in
cmmd-center/apex - After merge, run
./scripts/deploy-<host>.sh(Phase A) or auto-applies via CI (Phase C+) - Drift workflow will auto-draft a docs PR updating this page
Bearer keys
Section titled “Bearer keys”| Key | Where it lives | Purpose |
|---|---|---|
LITELLM_MASTER_KEY | ~/.apex/litellm/.master_key on Mac Studio + Ryzen | All-access bearer token for both LiteLLM proxies. Currently the only key that works (no virtual key support yet — needs Postgres backend). |
To generate a new master key:
NEW_KEY="sk-litellm-$(openssl rand -base64 36 | tr '+/' '-_' | tr -d '=')"ssh mac-studio-cf "echo '$NEW_KEY' > ~/.apex/litellm/.master_key && chmod 600 ~/.apex/litellm/.master_key"ssh ryzen-cf "echo '$NEW_KEY' > ~/.apex/litellm/.master_key && chmod 600 ~/.apex/litellm/.master_key"ssh mac-studio-cf 'launchctl kickstart -k user/$(id -u)/com.cmmd.litellm-apex'ssh ryzen-cf 'sudo systemctl restart litellm'Then update wherever clients store it (Forge/CMMD app configs).
Related
Section titled “Related”- Tool use verification — exact curl tests + Claude Code CLI smoke results (2026-05-19)
- Architecture — Mac Studio + Ryzen + Cloudflare tunnel topology
- API Reference — endpoint surface, headers, examples
- ADR-0003 — Apex Model Choices
- ADR-0002 — Anthropic-Compat Endpoints