Apex — Tool Use & Claude Code CLI Verification

Status as of 2026-05-19: apex-core and lab-flash pass tool use end-to-end. lab-general is broken for tool use (abliterated parser mismatch). Claude Code CLI works against apex-core via LAN-direct localhost; Cloudflare-tunneled endpoints fail with 524 due to 100s gateway timeout.

Results matrix

Alias	Backing model	Box	/v1/messages + tools	Claude Code CLI (LAN)	Claude Code CLI (via CF)
apex-core	qwen3.5:122b-a10b	Mac Studio	✅ tool_use block	✅ Verified end-to-end	❌ CF 524 (cold)
lab-flash	haervwe/GLM-4.6V-Flash-9B	Ryzen	✅ tool_use block	Expected ✅ (not exercised)	❌ CF 524 likely
lab-general	huihui_ai/Qwen3.6-abliterated:27b	Ryzen	❌ Emits raw text in thinking, parser misses it	❌ Not viable	❌ Also CF 524 + 4 min/turn

Test 1: Direct /v1/messages curl with tools array

This is the protocol-level test. If this fails, nothing else can work.

apex-core — ✅ PASS

export KEY="$LITELLM_MASTER_KEY"
curl -sS https://apex-api.cmmd.ai/v1/messages \
  -H "Authorization: Bearer $KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"apex-core","max_tokens":256,
       "tools":[{"name":"get_weather","description":"Get current weather for a city",
                 "input_schema":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}],
       "messages":[{"role":"user","content":"What is the weather in Tokyo? Use the get_weather tool."}]}'

Response (truncated):

{
  "stop_reason": "tool_use",
  "content": [
    { "type": "thinking", "thinking": "..." },
    { "type": "text", "text": "" },
    { "type": "tool_use", "name": "get_weather", "input": { "city": "Tokyo" } }
  ]
}

stop_reason: tool_use confirms the model properly emitted a function call. Anthropic SDK clients will pick this up and execute the tool.

lab-flash — ✅ PASS

Same curl, model lab-flash, endpoint https://lab-apex-api.cmmd.ai. Response:

{
  "stop_reason": "tool_use",
  "content": [
    { "type": "thinking", "thinking": "" },
    { "type": "text", "text": "I will call the get_weather function to find out the current weather in Tokyo." },
    { "type": "tool_use", "name": "get_weather", "input": { "city": "Tokyo" } }
  ]
}

GLM-4.6V’s native multimodal function calling works as advertised. Even better, it emits a useful preamble text before the tool call.

lab-general — ❌ FAIL

Same curl, model lab-general. After 4 min 24 s (most weights running on CPU because 27B doesn’t fit Ryzen’s 8GB GPU):

{
  "stop_reason": "max_tokens",
  "content": [
    {
      "type": "thinking",
      "thinking": "Thinking Process:\n1. Identify the user's request: weather in Tokyo.\n2. Identify the available tool: get_weather...\n... `{\"name\": \"get_weather\", \"parameters\": {\"city\": \"Tokyo\"}}` ..."
    },
    { "type": "text", "text": "" }
  ]
}

The model knew it should call the tool and wrote the tool call as plain text inside the thinking block. But:

1. The call uses OpenAI-style "parameters" key instead of Qwen-native "arguments"
2. There are no <tool_call>...</tool_call> markers around it
3. Ollama’s qwen3.5 parser doesn’t extract this as a structured tool_use block
4. The model hits max_tokens: 256 still in thinking mode before any structured output

Net: abliteration corrupted the tool-call template. Even with more tokens, the parser would not pick this up.

Test 2: Claude Code CLI end-to-end (apex-core via LAN-direct)

The protocol test only proves LiteLLM ↔ Ollama plumbing. The real test is: does Claude Code CLI itself work?

# On Mac Studio (LAN-direct, bypass Cloudflare)
mkdir -p /tmp/claude-apex-local && cd /tmp/claude-apex-local
touch hello.txt world.txt readme.md

export ANTHROPIC_BASE_URL="http://localhost:4000"
export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
export ANTHROPIC_MODEL="apex-core"
export ANTHROPIC_SMALL_FAST_MODEL="apex-core"
unset ANTHROPIC_API_KEY  # critical — otherwise Claude Code tries anthropic.com

claude --print --permission-mode bypassPermissions \
  "List the files in this directory using the LS tool. Just list filenames. After listing them, say DONE."

Output (after 8 min 23 s):

hello.txt
readme.md
world.txt

DONE

This proves the full loop:

1. Claude Code CLI sent a request with its system prompt + ~15 built-in tools (LS, Read, Write, Bash, Grep, Glob, Edit…)
2. LiteLLM proxied the Anthropic-format request to Ollama’s /api/chat with tools: [...]
3. Qwen 3.5 122B-A10B emitted a tool_use block calling LS with the cwd
4. LiteLLM translated back to Anthropic format
5. Claude Code CLI received the tool_use, executed the LS tool locally, captured the result
6. Sent the tool_result back as a follow-up message
7. Model received the tool output, generated final assistant text including “DONE”
8. Loop terminated cleanly

This is the canonical proof that apex-core works as a Claude Code CLI backend.

Test 3: Claude Code CLI via Cloudflare endpoint — fails

# Same setup but using the public endpoint
export ANTHROPIC_BASE_URL="https://apex-api.cmmd.ai"
# (everything else identical)
claude --print --permission-mode bypassPermissions "List files..."

Result:

API Error: 524 — The origin web server did not return a complete response
within the 120-second Proxy Read Timeout window.

Root cause: Cloudflare Pro plan has a 100s proxy read timeout. The first turn of a Claude Code session sends a ~5–10k token system prompt + tool schemas. On a cold model, processing exceeds 100s before the first response byte. Cloudflare 524s the connection.

Workarounds (in priority order):

1. LAN-direct from inside the network — set ANTHROPIC_BASE_URL="http://mac-studio:4000" if your laptop can reach Mac Studio over the office LAN or via Tailscale/WireGuard. No CF in the path = no timeout.
2. Pre-warm the model before invoking Claude Code: curl https://apex-api.cmmd.ai/v1/messages -d '{"model":"apex-core","max_tokens":10,"messages":[{"role":"user","content":"hi"}]}'. Once ollama ps shows the model resident, subsequent requests start faster.
3. Enable streaming — verify LiteLLM is configured to stream upstream from Ollama. Cloudflare’s 524 only fires when the origin is silent for 100s; if bytes are flowing the connection lives indefinitely.
4. Cloudflare Enterprise — extends timeout to 600s. ~$200+/mo. Not needed if streaming works.

Repeat-this-test recipe

When you add a new Apex model alias, run all three tests:

# Test 1: protocol-level tool use
KEY="$LITELLM_MASTER_KEY"
MODEL="<new-alias>"
ENDPOINT="<https://apex-api.cmmd.ai or https://lab-apex-api.cmmd.ai>"

curl -sS "$ENDPOINT/v1/messages" \
  -H "Authorization: Bearer $KEY" \
  -H "anthropic-version: 2023-06-01" \
  --max-time 360 \
  -d "{\"model\":\"$MODEL\",\"max_tokens\":256,
       \"tools\":[{\"name\":\"get_weather\",\"description\":\"Get weather\",
                   \"input_schema\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"]}}],
       \"messages\":[{\"role\":\"user\",\"content\":\"Weather in Tokyo? Use the tool.\"}]}" \
  | jq '{stop_reason, content: [.content[] | {type, name, text: (.text // "")[:80]}]}'

# Pass criteria: stop_reason == "tool_use" AND content contains a tool_use block

# Test 2: full Claude Code CLI loop (LAN-direct from the host running LiteLLM)
ssh <host> '
  cd /tmp && mkdir -p ct && cd ct && touch a.txt b.txt
  export ANTHROPIC_BASE_URL="http://localhost:4000"
  export ANTHROPIC_AUTH_TOKEN="'"$KEY"'"
  export ANTHROPIC_MODEL="'"$MODEL"'"
  unset ANTHROPIC_API_KEY
  claude --print --permission-mode bypassPermissions "List files in this directory with LS, then say DONE."
'

# Pass criteria: output contains the filenames AND "DONE"

Drop the results into the Models page under the alias’s row.

Why we test all three layers

1. Ollama capabilities (ollama show) — necessary but not sufficient. A model can declare tools capability but still fail in practice (lab-general).
2. Direct curl — proves LiteLLM + Ollama plumbing works at the protocol level.
3. Claude Code CLI end-to-end — proves the real user-facing surface works, including the full tool-call → tool-execute → tool-result → continue loop. This is the only test that catches things like “tool name collision” or “schema validation rejection” that protocol tests miss.

Related

• Models — per-alias capabilities matrix with verification status
• API Reference — endpoint surface
• Architecture — physical topology + Cloudflare tunnels
• ADR-0002 — Anthropic-Compat Endpoints — why we chose this protocol path
• ADR-0003 — Apex Model Choices — which models and why