Enterprise Evals 101
Imagine your worst nightmare: your agent passes every demo, then a customer asks something you never tested, and it fails in public.In April...
Topic
#eval-engineering
10 篇笔记
What FDE means at OpenAI
What forward deployed engineering means at OpenAI
How To Use Decision Models for Evals
Co-authored with Antaripa Saha.Decision models have exploded onto the scene recently, starting with TypeSafe’s Jev, followed by Perplexity’s...
为什么我们需要 Agent 评测
Agent 评测已经成为了当前 Agent应用的基石。如果说年初大家都在关注 Harness，那目前的热点和深水区，一定是 Agent评测和 Agent Infra。我所了解到的是，基本每个做 Agent 产品的组都很看重评测的方法论以及效果。因此我专门开了一个系列，给大家详细介...
The 5 levels of AI agents. From a single prompt to a production agent (Complete course)
There are five words people throw around about agents right now. Context engineering, loop engineering, Jev engineering, harness engineering...
How we Build Agent Environments & Tasks
TLDR: This is a practical guide on how we create synthetic agent environments and tasks. By the end, we have a two-step pipeline. The first...
7 rules for self-improving agent loops every AI engineer should know
By @Saboo_Shubham_ and @secchi_eliaWhen coding agents write the code, defining what's good is the real engineering job.Coding agents can now...
Eval Engineering: build the gate that lets your agents merge without you (full 6-step course)
The end state is easy to describe. An agent finishes a change, opens it, and it goes in without a human reading it.Not because anyone decide...
Evaluating code review agents with ReviewBench
There are more code review agents now, and we’ve been building one internally. Code review is hard to evaluate. There aren’t many benchmarks...
Towards Automating Eval Engineering
Today we’re releasing our Eval Engineering Skill, a skill that helps coding agents build evals using context from a repository and agent tra...