Function calling on DeepSeek, Qwen and Doubao, emulated

Coding agents need one API feature above all others: tools in, tool_calls out. The gateway that fronts DeepSeek, Qwen3 and Doubao for genmux drops the tools field on the floor — the models themselves were trained on function calling, the pipe between us and them simply does not carry it. So since v0.4.0 genmux emulates it: tool definitions are rendered into the system prompt, the reply is parsed back into standard OpenAI tool_calls, incrementally, so streams keep streaming. This post is the engineering behind that, the numbers, and what it means for the request you send.

The shape you send and get back

Nothing custom. The Python OpenAI SDK, unmodified:

from openai import OpenAI
client = OpenAI(base_url="https://api.genmux.tech/v1", api_key="nb_live_...")
tools = [{"type": "function", "function": {"name": "get_weather", "description": "Weather by city",
          "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
r = client.chat.completions.create(model="deepseek-v4-flash",
        messages=[{"role": "user", "content": "Weather in Paris?"}], tools=tools)
call = r.choices[0].message.tool_calls[0]        # call.function.name == "get_weather"
# run your function, then send the result back:
r2 = client.chat.completions.create(model="deepseek-v4-flash", tools=tools, messages=[
    {"role": "user", "content": "Weather in Paris?"},
    r.choices[0].message,
    {"role": "tool", "tool_call_id": call.id, "content": "18°C, cloudy"},
])

tool_choice takes auto, none, required and {"type":"function","function":{"name":…}}; streamed replies carry delta.tool_calls chunks with index, id and incremental arguments, ending in finish_reason: "tool_calls". Limits: 128 tools and 64 KiB of definitions per request.

Why <function_call> and not <tool_call>

The obvious design is to instruct the model in the format it was trained on. Qwen3 and DeepSeek chat templates use the Hermes convention — <tool_call>{"name":…,"arguments":{…}}</tool_call> for calls and <tool_response> for results — so run 1 of our eval rendered exactly that. It scored 35–60% on DeepSeek and 0–5% on Qwen and Doubao.

The replies explained why. <tool_call> and <tool_response> are special tokens in the Qwen tokenizer, and the gateway mangles them in both directions. Qwen3.7-Max saw our instruction as "within </think> tags" and answered with <tool_call_invoke> or <tool_call >; DeepSeek V4 Pro's opening tag was swallowed, leaving {…}</tool_calls> for the parser. The same prompt rendered three ways against the vendor directly confirmed it: the native tags were rewritten, and neutral tags came back verbatim from every model.

So the instructed protocol is Hermes-shaped with neutral names:

<function_call>
{"name": "write_file", "arguments": {"path": "hello.txt", "content": "hi"}}
</function_call>

and tool results go back as <function_result> blocks in a user turn (consecutive results collapse into one turn, because the vendor template does not like user/user pairs). Assistant turns that contained tool_calls are re-rendered as that text on the next request, so the model sees a consistent history whichever format it originally drifted into.

A parser that accepts the drift

Telling a model the format does not stop it from using its own. Over 20 tasks × 7 models we collected every shape they produce, and the parser accepts all of them:

• the instructed <function_call> block, and the native <tool_call> when the gateway lets it through;
• DeepSeek's <｜tool▁call▁begin｜> markers and its own nesting, <function name="x"><arguments>…;
• Anthropic-style <invoke name="x"><parameter name="k">v</parameter> — with argument types coerced from the tool's JSON schema, since XML has no numbers or booleans;
• ```tool_call fences, "parameters" or "input" instead of "arguments", arguments hoisted to the top level, the {"type":"function","function":{…}} wrapper;
• a stray } or ] after the block (Doubao Pro and V4 Flash each did this once in the run);
• a bare JSON call object as the whole reply, which counts as a call when tool_choice forced one or the object names a known tool.

Anything that still does not parse is emitted as text — the client sees the raw block instead of a broken request — and increments a per-model malformed counter. A rising counter on one model means new drift, and the fix is another parser case, not a redeploy of anything model-side.

Streaming is where this gets subtle. The parser is incremental: text is forwarded as soon as it cannot be the start of a marker, a block is buffered until it closes, and a reply that opens with { is held to the end in case it is a bare call. The net effect for a client is that "Let me check the file first" streams as content, and the call arrives as delta.tool_calls a moment later, exactly as OpenAI would send it.

Measured: 20 tasks, 7 models

The eval runs 20 agent-style tasks per model through the real HTTP API: weather lookups (including in Russian), prompts that must not call anything, required and named tool_choice, none, file writes with code, multi-line JSON, Unicode and a 12-line checklist, shell commands with quotes, nested array arguments, two calls in one turn, a full round trip (role: tool result, then prose) and one streamed write. Results from 2026-09-07:

Every model handles the basic loop — call, result, answer, streamed or not. The remaining misses are model behaviour: ignoring tool_choice: required, which we can instruct but not enforce. The gate for keeping a model's tools flag on is 90%; the three models without the flag (qwen3-6-flash, doubao-seed-2-lite, doubao-seed-2-mini) return 400 tools_unsupported, and GET /v1/models shows the flag as params_schema.tools.

JSON mode: instruct, validate, retry once

response_format came with the same release and works on the same principle, with one addition — validation. {"type":"json_object"} instructs; {"type":"json_schema","json_schema":{"name":…,"schema":{…}}} instructs and validates the non-streamed reply against the schema (JSON Schema 2020-12). If validation fails, genmux retries once with the validator's error fed back to the model; if that fails too, the request ends in 502 json_invalid and both attempts are billed, since both were generated. Streams are passed through unvalidated — there is nothing to validate until the stream ends, and by then it has been delivered. strict is accepted and ignored: there is no constrained decoding upstream. JSON mode and tools cannot be combined in one request.

What this means for your request

• finish_reason: "tool_calls" when at least one call was parsed, "stop" otherwise. The assistant message you get back can be appended to messages as-is, role: tool results included, exactly like OpenAI.
• Tool tokens are prompt tokens. The definitions are rendered as text, about 40 tokens per tool plus its schema, and billed at the model's input rate on every request. OpenCode's ~20 tools are roughly 1.5K tokens per request: on deepseek-v4-flash at $0.198 per 1M that is 0.024 credits≈ $0.0003per request; on qwen3-max at $1.08 it is about five times that. History dominates either way.
• required and named tool_choice are advisory. Design the loop so a prose answer is a soft failure — re-ask, or fall back to auto.
• One call per turn is typical. parallel_tool_calls is accepted and ignored; agents loop, so it does not matter in practice.
• Tool definitions and arguments are never stored — the same rule as message content. Job rows keep the count of tools, the tool_choice mode and usage.tool_calls, nothing else.

Getting started

Send tools to any of the seven models above and read tool_calls back; the OpenAI SDK needs nothing but a base_url. The client-side setups are in the posts on OpenCode, aider, Cline and Roo Code and Continue; the coding agents page has the one-screen summary, and the full parameter reference is in the docs.