vLLM Semantic Router Releases d3-ultra: A 27B Open Multimodal Decision Model

📝 摘要 #

vLLM Semantic Router 团队发布了 d3-ultra，一个 26.09B 参数、Apache-2.0 开源的多模态「决策模型」（Decision 3.0 家族从 0.6B 到 27B 的最新成员）。它不生成任何文本，而是读入一次输入（文本 / JSON / 图片 / 视频），对每个问题返回一个概率：支持 Choice、Yes/No、Score、Box、Track 五种题型。核心设计是 state once——每次请求只读一次输入，全部问题共享这一次读取，因此 255 个关于同一张照片的问题只需 2.16 秒（首问题 304ms，之后每个问题约 7.3ms）。架构基于 Qwen3.5 风格骨架，64 层中 48 层线性注意力 + 16 层全注意力，单题上下文 131,072 token（超限直接报 max_length_exceeded，不静默截断），已在单张 AMD Instinct MI325X 上以 BF16 验证。

在团队自建的 SR-Text（10,411 条文本请求 / 73,495 个问题，chance-corrected 计分）上得 87.2，领先 GLiDE V1.2（85.1）、d1-ultra（83.4）、d3-turbo（83.1）、d3（81.4）和 Jev 1.14（73.1），仅在「艺术与人类品味」一项略输 d1-ultra。SR-Box（5,678 个框选问题 / 4,984 张图 / 10 个领域，IoU≥0.5 判定正确）得 58.2%，低于 d3-turbo 的 60.0%——兄弟模型在自然图像更强，d3-ultra 在图表上占优。

核心概念： 把「路由 / 分类 / 视觉定位 / 视频核验」从多次生成式调用压缩成 1 次开放权重决策调用，用概率而非文本作为输出契约。

关键技术栈 / 关键要点：

1. 决策模型（非生成） — 输出是每个问题每个答案的概率，天然适配路由、闸门和分类场景，成本远低于让 LLM 生成并解析。
2. State once 架构 — 一次读取 + 多问题并行评估，把边际成本摊薄到 ~7ms/问题，是这一路线最大的经济杠杆。
3. 五类题型 + 多媒体输入 — Choice / Yes-No / Score / Box / Track；Box 恒返回单框 + 置信度，置信度趋近 0 即表示目标不在画面中。
4. 混合注意力骨架 — 64 层中 48 层线性注意力、16 层全注意力，兼顾长上下文与吞吐。
5. 四条部署路径 — Transformers（transformers==5.17.0，可选 flash-linear-attention）、vllm-sr serve -e vllm-sr/d3-ultra、decision.vllm-sr.ai 免费 API、官方 HF Space。

关键结论 / 实践建议： 文本侧可以直接考虑用 d3-ultra 替代更贵的 LLM-as-Judge / 路由调用；视觉框选场景则应先做 A/B，自然图像上 d3-turbo 仍更强。所有 benchmark 数字均为团队在私有测试集上自报，选型前建议在自己的 golden set 上复测。

The vLLM Semantic Router team has released d3-ultra, a 27B multimodal decision model under Apache-2.0. The d3-ultra decision model does not generate text. It reads an input once and returns a probability for every answer to every question. For developers, that means routing, classification, visual grounding and video checks from 1 open-weight call.

TL;DR #

• Size: 26.09B parameters (dense, including a 0.46B vision encoder); 131,072 tokens of context per question.
• Runs on: tested on 1 AMD Instinct MI325X with BF16 weights; no official quantized variant is listed yet.
• Performance: highest SR-Text score of the 6 decision models in the team’s own private test.
• Best: 97.8 on SR-Text tools and automation (chance-corrected accuracy).
• Worst: 54.6% on SR-Box (9 domains), 5.0 points below its sibling d3-turbo.

What is d3-ultra? #

d3-ultra is a System One decision model from vLLM Semantic Router that answers typed questions with probabilities. It is the newest member of Decision 3.0, a family that spans 0.6B to 27B.

TEXT TABLE LAYOUT Is your PDF parser actually getting it right?omni_parse_bench · scores text, tables & layout · open sourceStar on GitHub → Sponsored

You send a state (text or JSON, plus images or videos) and a set of questions. Each question has a type:

• Choice: pick 1 of several options.
• Yes / No: return the probability of yes.  
• Score: rate on a scale.
• Box: locate something in an image or a video moment.  
• Track: follow a target across a whole video.

Images can also sit inside questions and options. A quiz can show a photo, then   offer 4 etchings as answers.

How does d3-ultra work? #

The core idea is state once. The model reads the input once per request and answers every question from that single read.

The config shows a Qwen3.5-architecture backbone with 64 layers. 48 are linear-attention layers and 16 are full-attention layers. The card does not name its base checkpoint. vLLM’s recipe says the sibling d3 was trained from Qwen3.8-27B.

100K200K300K+users hand AI their chores.+30% wallet bonus · ends Oct 31Claim 30% bonus → Sponsored

Box answers always return exactly 1 box with a confidence. A confidence near 0 means the target is not in the picture. Questions over 131,072 tokens fail with max_length_exceeded; nothing is silently truncated.

How does d3-ultra perform on benchmarks? #

The research team compares d3-ultra with GLiDE V1.2, d1-ultra, d3-turbo, d3 and Jev 1.14 on 2 in-house tests.

SR-Text has 10,411 text requests with 73,495 questions. Most recreate skills from 34 public benchmarks with new items. Another 4,896 questions come from 43 new task families, like SQL checks and ticket routing. Scores are chance-corrected: 0 is chance, 100 is perfect.

• d3-ultra: 87.2
• GLiDE V1.2: 85.1
• d1-ultra: 83.4
• d3-turbo: 83.1
• d3: 81.4
• Jev 1.14: 73.1

d3-ultra leads every area except arts and human taste. There, d1-ultra scores 90.8 against 89.6, within the margin of error.

SR-Box has 5,678 Box questions on 4,984 images across 10 domains. A box counts as correct at IoU 0.5 or higher. d3-ultra scores 58.2% over 10 domains and 54.6% over 9. d3-turbo scores 60.0% and 59.6%. Jev 1.14 scores 45.7% and 46.9%. The team says d3-turbo stays stronger on natural images, while d3-ultra leads on charts.

Latency: on 1 AMD Instinct MI325X with AutoModel, 1 question about a photo takes 304 ms. 255 questions take 2.16 s, about 7.3 ms per extra question.

How do you deploy d3-ultra? #

There are 4 routes:

• Transformers: AutoModel.from_pretrained("vllm-sr/d3-ultra", trust_remote_code=True) with transformers==5.17.0. The optional flash-linear-attention package adds fast GPU kernels.
• vLLM Semantic Router Engine: vllm-sr serve -e vllm-sr/d3-ultra, added in PR #5449 on October 25, 2026.
• Free API: the card says decision.vllm-sr.ai serves d3-ultra, with a free key.
• Demo: the official Hugging Face Space.

d3-ultra vs GLiDE V1.2, d1-ultra, Jev 1.14 and d3-turbo #

Key Takeaways #

• d3-ultra is a 27B Apache-2.0 decision model that never generates text.
• It reads input once: 255 questions about 1 photo take 2.16 s.
• It tops the team’s SR-Text test at 87.2, ahead of GLiDE V1.2 at 85.1.
• It trails d3-turbo on Box grounding: 58.2% vs 60.0%.
• All benchmark numbers are self-reported on private test sets.

Check out the model card, the GitHub repo, the Decision 3.0 collection and the Hugging Face Space. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.