How to Cut Your Agent Bill by 90% With Jev Engineering and Kimi K3

📝 摘要 #

本文介绍 TypeSafe AI 推出的 Jev 模型及其"Jev 工程"实践方法论——一种用只做决策（不生成文本）的专用模型替代昂贵 frontier LLM 处理 agent 循环中海量"小决策"的降本方案。Jev 接收状态和结构化问题，返回选项/评分/是与否判断及校准置信度，单次调用约 100ms、$0.042/百万输入 token（输出免费），在 TypeSafe 自测中比通用 LLM 快 193.6 倍、便宜 444.6 倍。文章完整阐述了四步分类法、三种问题原语、单调用 speculative fan-out、提问编写规则、已知弱点及 harness 修复，以及 Jev→Kimi K3→人类的级联路由架构。

核心概念： Agent 循环中绝大多数 token 消耗在非创造性"小决策"上（路由、分类、安全判断、完成判定），用一个经 RLCD 训练、只输出校准概率的专用模型替代这些决策调用，可将账单降低约 90%。

关键技术栈 / 关键要点：

1. Jev 模型（jev-1.13.0） — TypeSafe AI 的专用决策模型，RLCD 训练追求诚实置信度（90% 置信度的答案约 90% 正确），只返回 choice/score/noul + probability，70-500ms，输出免费
2. 三种问题原语 — Choice（≤255 选项，概率和为 1）、Score（2-10 级有序评分，可落在级别之间）、Noul（是/否，0.5 表示无法判断）
3. Speculative fan-out — 同一 state 下的多个问题在单次请求中并行评估，13 个问题合一调用比 13 次分开便宜 12.2 倍、快 10 倍
4. 级联路由架构 — Code（精确规则，免费）→ Jev（高于阈值的决策）→ Kimi K3（低置信度及需书面理由的案例，1M 上下文 $3/$15）→ 人类（不可逆操作）
5. 提问即技能 — question ID 不发给模型、一问一判断、描述情境而非程度、每个 Choice 必须有 other 退出项、Noul 高值即 yes、用反引号路径指向具体字段
6. 已知弱点与 harness 修复 — 字面理解、非计算器（计数在 code 做）、日期是文本、context rot、信任 state（对抗性文本）、不能解释/看图、多跳推理退化，均用确定性代码 + 第二模型兜底
7. 框架集成 — 已内置于 LangChain（ModelRouterMiddleware/AutoModeMiddleware）、Pydantic AI（类型→问题自动映射 + FallbackModel）、Vercel AI SDK
8. 真实案例 — Hassan 欺诈筛查 96/100 正确 16 秒 $0.07、Browser Use 航班搜索 $0.0039、1kpapers 1018 篇分类 $0.08、awlevin 桌面操作 $0.0002/决策 vs Opus 5 的 $0.032

关键结论 / 实践建议： 先用一个 agent transcript 做四桶分类，找出每天重复 50 次的单一决策移到 Jev；按 action 而非按 model 设阈值（人类审查地板 0.5，破坏性操作 ≥0.9）；运行一周 shadow mode 对比置信度分带；pin 版本、全量日志、把问题当代码版本化管理。1 万邮件/月场景下 Jev+K3 级联成本 $9.42 vs frontier LLM $300。

How to find the decisions hiding inside your agents, write questions a system one model answers well, route on calibrated confidence and hand the hard cases to Kimi K3. The API, the code, the weak spots and the math.

Open any agent transcript and count the moments where the model wasn't creating anything. Which tool next. Is this source relevant.

Did the test pass. Is this command safe. Is the task done. In most loops those small calls outnumber the real work, and every one of them is billed as a full generation from a frontier model.

On September 15 TypeSafe AI came out of stealth with a model built only for those calls. Jev never writes a word.

You send it state and typed questions, it sends back answers with probabilities, usually in around 100 milliseconds, at $0.042 per million input tokens with output free.

On TypeSafe's own workflow evals it came out 193.6x faster and 444.6x cheaper than the LLMs it was measured against, and the company itself calls those numbers the high end of what to expect.

Jev engineering is the craft of using it well: finding the hidden decisions, writing questions the model can answer, routing on its confidence, and knowing where it breaks. This is the full playbook, with Kimi K3 as the model that takes the cases Jev isn't sure about.

1/ Why a Model That Can't Write Is Useful #

Diogo Almeida co-authored InstructGPT, the RLHF work that turned GPT-3 into ChatGPT. His argument for Jev starts with a critique of his own technique: models trained to please human raters learn to look right, and their stated confidence stops meaning anything. If a model can do a task 95% of the time but can't tell you which 5% it's getting wrong, you can't automate that task.

Jev is trained with what TypeSafe calls RLCD, reinforcement learning for calibrated decisions. The target is honest probabilities: answers given 90% should be right about 90% of the time across your data.

Individual answers can still be wrong, which is exactly why the confidence number matters. It tells your code which answers to act on and which to send somewhere else.

A real run shows the idea. Hassan (@nutlope) screened 100 emails, half legitimate and half fraud. Jev classified all 100 in 1.42 seconds. The 31 answers under 95% confidence went to Kimi K3 for a full read. The pipeline finished 96 of 100 right in 16 seconds for about $0.07, and Jev's share of that bill was a third of a cent.

2/ Step One: Sort Every Step of Your Agent #

Go through one real transcript and put every step in one of four buckets.

The third bucket is always bigger than people expect, and it's the one Jev engineering moves. Everything else stays where it is.

3/ The Primitives and the API #

Three question types cover the third bucket.

The facts you'll need while building:

• Endpoint: POST https://api.typesafe.ai/v1/systemone with state, model and questions. Python SDK: pip install typesafe-sdk, then client.system_one(...)
• Model: pin jev-1.13.0 once your thresholds are tuned. jev-latest moves.
• Size: state plus all questions share about 64K tokens, and state plus the longest single question must fit in about 32K (roughly 150,000 characters of English).
• Input: text only. Strings, JSON objects or arrays, no images.
• Rate limits: 1,200 requests and 250,000 tokens per second on jev-1.13, and TypeSafe says those will move as capacity lands.
• Speed and price: 70 to 500 ms end to end, $0.042 per million input tokens, output free. 10,000 decisions at 1,000 tokens each cost $0.42.

4/ Ask Everything in One Call #

Questions in a single request are evaluated in parallel against the same state.

TypeSafe calls this speculative fan-out, and it's the single biggest cost lever you control. In one of their tests, 13 questions over a 53,777-character document ran 12.2x cheaper and 10x faster as one call than as thirteen, mostly because the state was sent once.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient(model="jev-1.13.0")

r = client.system_one(
    state={"ticket": ticket_text, "customer": {"plan": "pro", "refunds_90d": 0}},
    questions={
        "department": Choice(
            instructions="Which team should own the problem described in \`ticket\`?",
            criteria={
                "billing": "Charges, invoices, plan changes",
                "bug": "Something in the product does not work as documented",
                "howto": "The customer asks how to do something that already works",
                "other": "None of the above fits",
            }),
        "frustration": Score(
            instructions="How frustrated is the customer, judged from \`ticket\` only?",
            criteria=[
                "Neutral or positive, no complaint about the experience",
                "Annoyed but polite, describes wasted time",
                "Angry, mentions cancelling, a public review or a chargeback",
            ]),
        "refund_requested": Noul(
            instructions="Does \`ticket\` ask for money back, a refund or a chargeback?"),
    },
)

dept = r.answers["department"]            # .choice  .probabilities  .confidence
mood = r.answers["frustration"].score     # can land between levels, e.g. 1.3
refund = r.answers["refund_requested"].noul

Two boundaries apply. Questions in one call can't read each other's answers, so if a decision depends on new evidence, fetch it first and ask again. And fan-out only pays when the questions share the state; unrelated questions belong in separate calls

5/ Writing Questions Is the Actual Skill #

Most bad Jev answers trace back to a bad question. These rules come from TypeSafe's docs and from people who shipped with it.

• The question ID is never sent to the model. Naming a field is_fraud tells Jev nothing. The meaning has to live in instructions and criteria.
• Ask one judgment per question. "Is this urgent and from a paying customer" is two questions. Split them and combine the answers in code.
• Describe situations, not degrees. "Mentions cancelling or a chargeback" is checkable. "Very frustrated" is a mood, and so are bare numbers as levels.
• Write every level so it stands alone. Levels are judged independently, so "worse than the previous level" means nothing to the model.
• Phrase every Noul so a high number means yes. A Noul where true means "not safe" is a bug waiting to happen.
• Always add an exit. other or none on every Choice, because the model has to pick something and an exit gives uncertainty a place to go.
• Point at the exact field. Backticked paths like `order.charges` tell Jev which part of the state a question is about.
• Send evidence, and filter in code first. Accuracy drops as irrelevant state piles up. Seven sources with their claims beat "the research looks mostly done," and a search result trimmed to the relevant rows beats the whole page.
• Turn extraction into selection. Jev can't produce a value that isn't in your schema. If you need to pull a vendor name from an invoice, give it the list of known vendors and ask it to pick one.

6/ Know Where It Breaks #

TypeSafe publishes a "jaggedness" page for jev-1.13. Every item on it has a harness fix.

The last line of defense is the same for all of them: type safety means Jev can't return a value outside your schema, but it can still return the wrong valid value. Confidence thresholds and a second model are how you catch those.

7/ The Cascade: Jev First, Kimi K3 on the Hard Cases #

Put it together and every decision falls through four layers, stopping at the first one that's sure.

Kimi K3 fits the second slot for practical reasons. An escalated case often needs the whole thread and history, and K3's 1M-token context is priced the same at any length. It reads the screenshots and scanned PDFs that Jev can't. Its weights are open, so the same code runs against Moonshot's API, Together (Hassan's setup) or your own host.

import json, os
from openai import OpenAI
from typesafe_sdk import Choice, TypeSafeClient

CRITERIA = {
    "customer": "A current customer asking about an order, a bill or their account",
    "lead": "Someone asking about prices, a demo or working together",
    "vendor": "Someone selling a product or service to us",
    "fraud": "Urgent payment request, changed bank details, spoofed sender or a login link",
    "other": "None of the above fits",
}
THRESHOLD = 0.95   # Hassan's setting for fraud, tune it per decision

jev = TypeSafeClient(model="jev-1.13.0")
kimi = OpenAI(api_key=os.environ["KIMI_API_KEY"],
              base_url=os.environ.get("KIMI_BASE_URL", "https://api.moonshot.ai/v1"))

def ask_kimi(email, jev_guess):
    reply = kimi.chat.completions.create(
        model="kimi-k3",
        response_format={"type": "json_object"},
        messages=[{"role": "user", "content":
            f"Classify this email as one of {list(CRITERIA)}. "
            f"A fast classifier guessed '{jev_guess}' with low confidence. "
            'Answer as JSON: {"category": "...", "reason": "...", "sure": true}\n\n' + email}],
    )
    return json.loads(reply.choices[0].message.content)

def route(email):
    r = jev.system_one(
        state={"email": email},
        questions={"category": Choice(
            instructions="What does the sender of \`email\` want?", criteria=CRITERIA)},
    )
    a = r.answers["category"]
    if a.confidence >= THRESHOLD and a.choice != "other":
        return {"category": a.choice, "by": "jev", "confidence": a.confidence}
    k = ask_kimi(email, a.choice)
    if k.get("sure"):
        return {"category": k["category"], "by": "kimi-k3", "reason": k["reason"]}
    return {"category": "review", "by": "you", "jev": a.choice, "kimi": k["category"]}

other never routes on its own. K3 gets Jev's guess as context. And when both are unsure, you get both answers side by side, which is the fastest review there is.

8/ Plug It Into the Harness You Already Have #

You don't need a new framework. Jev already ships inside the three most common ones.

LangChain (langchain-typesafe) has two middlewares. ModelRouterMiddleware lets Jev pick the cheapest model that can handle a request, against criteria you write.

AutoModeMiddleware checks every tool call for risk before it executes, the kind of dangerous-action classifier coding harnesses have kept closed-source until now:

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import AutoModeMiddleware

agent = create_agent(YOUR_MODEL, middleware=[AutoModeMiddleware(tools=["bash"])])

Pydantic AI (pydantic-ai-slim[typesafe]) turns your output type into questions: Agent("typesafe:jev-latest", output_type=Triage).

A bool becomes a yes or no, a Literal becomes a Choice, an IntEnum with a docstring on each member becomes a Score, and X | None adds a "None of these" option. Its FallbackModel pattern hands unsure answers to an LLM, which is the cascade above in two lines.

Vercel AI SDK exposes Jev through experimental_evaluate with the @ai-sdk/typesafe-ai provider, and through the AI Gateway as typesafe-ai/jev at the same price.

9/ What People Are Already Running #

The pattern repeats: the menu of options is rebuilt from live state at every step, Jev picks, and code or a small model does the rest.

10/ Jev Steering a Kimi K3 Swarm #

The same idea scales up. When the job is big, Kimi K3's Agent Swarm does the building with up to 300 sub-agents, and Jev handles the between-turn calls: which role runs next, which returns to keep, and whether the goal is met.

Three Jev calls per turn cost a fraction of a cent, so the loop can check "are we done" after every turn and stop itself. When the done-check lands in the middle band, one K3 call with the full state decides.

11/ Thresholds and Rollout #

Set the threshold per action, not per model. TypeSafe's docs use 0.5 as a floor for human review and 0.9 before anything destructive, and call them examples rather than defaults. Filing a newsletter can go automatic much lower than flagging fraud.

Run a shadow week. Let Jev label while the old path decides. Then compare by confidence band: if the 0.9+ answers match yours nearly every time, that band goes automatic first.

Pin the version and log everything. Model ID, full distribution, threshold, which layer answered, what happened next. The log becomes your evaluation set, and a pinned version means your thresholds still mean the same thing next month.

Treat questions like code. A change to criteria changes production behavior. Version it, test it on the labeled set, roll it back when it drifts.

12/ The Math #

10,000 emails a month. Assumptions: about 1,000 input tokens per email for Jev, 1,500 in and 300 out for a full model call, 10% escalated.

TypeSafe's own cascade estimate lands in the same place at scale: about $6,480 instead of $30,400 for a million support tickets.

13/ What It Pays #

The third one is the fastest to sell. Every team running agents is paying frontier prices for decisions, and the audit shows them the exact calls to move.

The Short Version #

Sort every step of your agent into create, exact rule, pick from known answers, and irreversible. Move the third bucket to Jev, ask every related question in one call, write criteria as situations with an exit option, and route on confidence per action. Send what Jev isn't sure about to Kimi K3, and keep the irreversible steps for yourself.

Start with one decision your agent makes fifty times a day.

The stack:

📁 Jev by TypeSafe AI ↳ https://typesafe.ai

📁 Jev API docs ↳ https://docs.typesafe.ai/api

📁 Kimi K3 API ↳ https://platform.kimi.ai

📁 Hassan's Jev + Kimi K3 fraud pipeline ↳ https://github.com/Nutlope/jev-fraud

And if you found this useful: #

• Bookmark this article. The links change and new repos pop up weekly, you'll need this as a reference
• For weekly deep dives into AI architecture, quant trading, and the agent economy, follow me:@polydao
• - here I share my raw prompts, custom skills, and alpha that's too early for XJoin the TG Channel:Buzzoni Notes