Enterprise Evals 101
Imagine your worst nightmare: your agent passes every demo, then a customer asks something you never tested, and it fails in public.In April...
Topic
#LLM
67 篇笔记
The era of personal software
📝 摘要 作者以自己构建的健康数据追踪应用为例，讨论 LLM 时代「个人软件（personal software）」的兴起：当构建成本足够低，软件将逐渐商品化，越来越多人会为自身需求量身定制工具。他完整记录了实现链路——Apple Health 数据经 Health Auto Export 每小时导出 JSON，通过 端点写入 $5 的 PlanetScale Postgres；数据库 sche…
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.
十分钟精读 DSH Super Agent：从 Paper 到 PRD
1. Super-agent作者认为，以 Coding Agent 为代表的单任务 Agent 已逐渐能够独立完成复杂工作，但现有系统仍以单一 Session 为基本单位。用户并行推进多个任务时，需要自行切换会话、管理上下文；文章称一个人通常只能同时跟进 3～5 个活跃 Sess...
Exploring Super-agent Design: Towards a More Autonomous Agent System
The original article is in Chinese. This is a machine translation.This article discusses architectural ideas. It is not an introduction to e...
New context engineering challenges in late 2026
It feels like everyone is converging on the same agent architecture: an LLM, running in a loop, calling tools, running in a sandbox. Claude ...
bliki: LLM Voice
The uncanny valley of LLMs' prose
vLLM Semantic Router Releases d3-ultra: A 27B Open Multimodal Decision Model
vLLM Semantic Router released d3-ultra, a 26.09B Apache-2.0 multimodal decision model scoring 87.2 on its own SR-Text benchmark.
Lemo-Opuscar 史诗级更新：8种新风格+新改编玩法，我把风格库变成了风格制作工具箱
Hello 大家好，我是lemomo。这次 Lemo-Opuscar 迎来了上线以来最大的一次更新，不光有新的风格，还有新的玩法，整个项目定位从【视频风格库】变成【搭建自己风格库的工具】。新增【剪辑与纪实】8 种风格，目前从最早的39种已经更新到了59种。未来会继续更新，我的目标...
LLMs are making formal software verification feasible at a much larger scale
Claude formally proved security properties in the compiled code of 123 open-source libraries. Along the way, it found bugs that had eluded detection for years.
How To Use Decision Models for Evals
Co-authored with Antaripa Saha.Decision models have exploded onto the scene recently, starting with TypeSafe’s Jev, followed by Perplexity’s...
形式化验证 LLM 生成的代码（FATTW 2026, Lucas Cordeiro）
FATTW 2026 会议演讲（与 ASE 2026 联合举办）：LLM 负责猜测（不变量、契约、harness），证明引擎负责裁决（ESBMC/k-induction/Vampire）。以 310,531 个 C 程序基准、Test-Comp/SV-COMP 竞赛成绩和 AWS/NVIDIA/vLLM 生产代码漏洞为证据，论证『模型建议 + 证明引擎确认』的设计原则。
「万字总结」- 我偷来了一份成为 Agent 工程师的学习线路
我之前面试了一家做 AI + web3 业内比较厉害的公司，但是没有通过吧，面试之后面试官给我了一份《Agent工程师入门指南》。我把它整理了一下，分享出来给大家提供参考，这是一份很详细很详细的、关于如何成为 Agent 工程师的文档，并且补充上我自己的学习心得当时给我的答复是 ...
How to Cut Your Agent Bill by 90% With Jev Engineering and Kimi K3
How to find the decisions hiding inside your agents, write questions a system one model answers well, route on calibrated confidence and han...
Jev 模型调研 — TypeSafe AI 的 System One 模型
Jev 模型调研 — TypeSafe AI 的 System One 模型 [!info] 一句话概括 Jev 是 TypeSafe AI 于 2026-09-14 发布的首个 System One 模型。它不生成文本，而是输入非结构化状态、输出带校准概率的类型化决策——可以理解为"一个具有前沿智能的函数调用"。 1. 背景 - 公司：TypeSafe AI，隐身（stealth）两年后公开…
KV Cache Engineering for LLM Serving, clearly explained
Everything you need to understand why the KV cache grows, the 12 ways models and serving engines reduce it, what each technique actually sav...
SemiAnalysis 发布 AgentX InferenceXv3 研报，Mooncake 成为关键基础设施
SemiAnalysis 发布 AgentX InferenceXv3，用大规模真实 Agentic workload 重新审视推理系统性能。随着长上下文、多轮交互和 KV Cache 复用成为常态，Agentic Inference 正从“拼算力”走向系统级优化，而 Mooncake 也正在其中扮演越来越重要的基础设施角色。
From context to dreams: architecting memory for AI agents
Have you ever felt that every conversation you have with an LLM across sessions feels like starting over from scratch? LLMs have a problem: ...
What Is Reasoning
A short summary of how reasoning traces work.
Optimizing for cost and intelligence
Balance cost and intelligence on the Claude Platform, with measured results for prompt caching, effort, model choice, budgets, and multi-model strategies.
How Production LLMs Reason Better at Inference Time (as researched by Google, OpenAI, and Anthropic)
8 techniques to make an LLM reason better at inference time, covered with tradeoffs and practical notes. Every one of them is running in pro...
Why model routing must be in the harness
Inference spend is rising as agents are deployed more widely, take on more complex tasks, and run longer autonomously. Enterprises are start...
How Compaction Works in Pi
Why compaction is needed for large language models and how Pi implements it
DeepSeek 刚支持了 OpenAI Responses API，但你可能还不知道它是什么
2026 年 7 月 31 日，DeepSeek 悄悄发布了 V4-Flash-0731，附带一行更新日志：原生支持 OpenAI Responses API。消息不大，但背后藏着一条值得认真讲的故事线。一套 API 格式，是怎么从聊天框时代的产物，一步一步演化成今天 agent...
Profiling LLM Inference with SGLang and Torch Profiler
This blog is my attempt to learn how to profile an llm served using sglang and its in built torch profiler integration that comes out of box...
Efficient Decode Context Parallelism with vLLM for Long Context Workloads
Decode Context Parallelism (DCP) in vLLM shards KV cache across GPUs by sequence dimension, enabling 3× higher throughput on long-context agentic workloads comp
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
前OpenAI推理负责人JT与Gemini pretrain负责人Rohan: Continue learning的未来
📝 摘要 本文是红杉资本对 Core Automation 联创 Jerry Tworek（前 OpenAI o1 reasoning 负责人）和 Rohan Anil（前 Google Gemini pretrain 负责人）的访谈。两人共同阐述了 Transformer 架构的根本局限——训练停止后无法真正持续学习——以及 Core Automation 寻找"Transformer 替代者…
A practical guide to building agents
A comprehensive guide to designing, orchestrating, and deploying AI agents—covering use cases, model selection, tool design, guardrails, and multi-agent patterns.
多模型路由进入 Agent 会话之后
模型路由在单轮问答中可以按难度分配成本，但进入 Agent 多轮会话后，还要处理上下文、工具状态、缓存和隐式状态。本文拆解会话级路由的智能与工程瓶颈，以及固定强模型、隔离子任务和有状态路由三条现实路线。
22580: From GPT2 to Kimi3, Explained
Twenty-two thousand five hundred and eighty. That’s how many GPT-2 (2019) models fit inside KimiK3 (2026). We scaled up by a factor of 22,58...
Kimi K3 Is Here: Efficient Day-0 Support on vLLM
vLLM delivers day-0 Kimi K3 serving with hybrid KDA prefix caching, DSpark speculative decoding, production-scale disaggregation, and optimized kernels across N
Practical multi-agent orchestration in Codex
GPT-5.6 Sol gets especially interesting when it has a team to work with. Codex's new Multi-Agent V2 tools give Sol and Terra a natural way t...
The new rules of context engineering for Claude 5 models
I’ve written previously about how to best prompt the newest generation of Claude 5 models and work with them iteratively to discover what yo...
大模型是专利期十个月的制药生意
这几天流传的 DeepSeek 梁文锋讲话里有一个有意思的数据点，正好和我最近的思考交汇，因此写下一些关于整个大模型产业的一些预测，供以后证伪修正思考。他说，DeepSeek 的 API 定价标准是：买一批设备回来，十个月收回成本。也就是十个月赚来的利润可以覆盖硬件上的投入。顺带...
The State of Agent Wikis
In April 2026, Andrej Karpathy published a GitHub Gist describing a pattern he called the LLM Wiki.In the months since, four different teams...
Controlling Reasoning Effort in LLMs
How LLMs Learn Low-, Medium-, and High-Effort Reasoning Modes
数据研发Multi-Agent架构的Harness工程实践
📝 摘要 本文基于阿里云数据研发团队的 Multi-Agent 实践，系统阐述了 Harness Engineering（智能体脚手架工程）的核心理念与落地方法。文章指出 AI 工程经历了 Prompt Engineering → Context Engineering → Harness Engineering 三代演进，并提出核心公式 Agent = Model + Harness：模型决定…
A Swarm of Agents for Multi-Angle Analysis: Building a Team of Experts from LLMs
Not about speed. About making several agents with different viewpoints argue over one decision and reach a conclusion better than any of the...
DSLs Enable Reliable Use of LLMs
Why Domain-Specific Languages and good abstractions make LLM code generation reliable and reviewable.
Your KV Caching Is Broken
A practitioner's guide to KV cache management. How a modern caching architecture cuts input token costs by 90% and speeds up LLM inference b...
Accelerating Gemini Nano models on Pixel with frozen Multi-Token Prediction
📝 摘要 Google Research 发布了一种将 Multi-Token Prediction (MTP) 增层改造（retrofit）到已冻结的 Gemini Nano v3 生产模型上的方法，在不修改基座模型权重的前提下，大幅加速 Pixel 设备上的端侧 LLM 推理。该方法已在 Pixel 9 和 10 系列实机部署，使 AI Notification Summaries、Proo…
Optimizing distributed AI inference: Advanced deployment patterns
Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.
vLLM + LMCache: A Starter Guide, No GPU Required
Get started easily: a single MacBook is all you need to develop vLLM + LMCacheFor New Contributors · Covering Frontend / L1 Eviction / L2 Storage / Observability If you ever skipped LMCache because you didn’t have a GPU on hand, this guide was written for you. LMCache’s multi-platform framework has already decoupled the GPU […]
Designing distributed AI inference: Core concepts and scaling dimensions
Learn about the five-dimensional design space in modern LLM serving, including tensor, pipeline, expert, data, and context parallelism.
Migrating LiteLLM to Rust - Building the Fastest and Litest AI Gateway
LiteLLM is moving its AI gateway to Rust: 15x throughput, 11x less memory, and sub-1ms per-request overhead. No v2, no migration, your config stays the same.
Qwen Studio
Qwen Studio offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.
The 7 Types of Agent Memory: A Technical Guide for AI Engineers
Agent memory explained: the 7 types — working, semantic, episodic, procedural, retrieval, parametric, prospective — and when to build each.
GLM-5 系列模型演化历程
GLM-5 系列模型演化历程 智谱AI（Z.ai）的 GLM 系列从 GLM-5 到 GLM-5.2 经历三次重大迭代，每次均有明确的技术突破点：架构基础 → Agent 持久性 → 极致长上下文与编码效率。本文基于多源深度调研（101 个子代理、19 个源、94 条 claim、3-投票对抗验证）梳理完整演化路径。 Open: Pasted image 20260620141932.png30-…
AI 101: What is Recursive Self-Improvement?
How AI systems are beginning to automate coding, experiments, evaluation, and research workflows ‒ and why Anthropic, Recursive, and Sakana ...
Building Reliable Agentic AI Systems
AI helping pharmaceutical researchers query decades of information buried in PDF reports
Advanced Observability for Production Systems
From SLIs to LLM monitoring: 9 practices that separate mature teams from dashboard collectorsMost teams have dashboards. Few have observabil...
Economy of Minds: Multi-Agent Prompt Optimization explained
Wake up babe there's a new paper from Harvard: Economy of Minds (EOM). They've made a decentralized multi-agent system where agents coordin...
Faking a Filesystem For your Agent: PostgresFS
Faking a filesystem for your agent: why PostgresFS lost to a skillCo written by @SufjanFana Working with agent harnesses, it is hard not to ...
How to Build AI Agent Swarms (Complete Guide)
This is a complete A–Z breakdown of AI Agent Swarms what they are, how to use them.Why they change everything about how you work with AI.B...
Dreaming: Better memory for a more helpful ChatGPT
ChatGPT introduces a new memory system to better remember preferences, keeping context fresh and relevant across conversations.
Session-Aware Agentic Routing: Continuity-Aware Model Selection for Long-Horizon LLM Agents
How Session-Aware Agentic Routing in vLLM Semantic Router preserves long-horizon agent continuity with session memory, safe model-switch boundaries, prefix-cach
从 305 GB 到 7.4 GB：大模型 KVCache 架构演进全景
五一本打算更新一年前写的开源引擎 KVCache 实现文档，但梳理下来发现，光是 KVCache 演进的理论部分就已足够撑起一篇独立文章。在 Agent 改变搜索习惯、重塑软件行业的今天，多轮交互带来的长上下文让 KVCache 持…
NanoWhale 项目深度解析：从零学习 DeepSeek-V4 架构
NanoWhale 项目深度解析：从零学习 DeepSeek-V4 架构 一句话总结：NanoWhale 是 HuggingFace 发布的一个教学项目，它用约 1.1 亿参数（110M）实现了一个"迷你版" DeepSeek-V4 大语言模型，从零开始训练，让你能用一张 H100 显卡就跑通完整的大模型训练流程。 一、这个项目在做什么？ 大模型训练的一般流程 训练一个大语言模型（LLM）通常分为…
Notes from inside China's AI labs
Lessons from my trip to talk to most of the leading AI labs in China.
罗福莉访谈里那几句关于 memory 的话，被几乎所有人忽略了
张小珺前几天发了一期跟罗福莉的长访谈，三个半小时。小宇宙，YouTube，B 站都有，外面已经有很多解读。大家讨论得最多的是 OpenClaw 怎么改变了她的世界观，1T 是不是入场券，组织平权怎么搞，3:1:1 的卡分配比例。听了以后，挑出来想单独写一篇的，是这期里被低估最严重...
State of Memory in Agent Harness
Agent harnesses are where AI software actually runs. Cursor, Devin, Claude Code, Codex: these environments handle context, orchestrate tools...
TurboQuant：谷歌KV Cache压缩算法深度解析
谷歌研究院于2026年3月发布TurboQuant算法，运用极坐标量化与JL变换技术，实现大模型KV Cache六倍压缩及注意力计算八倍加速，且保持精度无损。
KV Caching in LLMs, Clearly Explained
You must have seen it every time you use ChatGPT or Claude that the first token takes noticeably longer to appear. Then the rest stream out ...
Inside vLLM: Anatomy of a High-Throughput LLM Inference System
From paged attention, continuous batching, prefix caching, specdec, etc. to multi-GPU, multi-node dynamic serving at scale.
Editing Files at 1000 Tokens per Second
A new model and inference method for high-accuracy full-file edits at 1000 tokens/s.