CheinTian / notes
Clipping · 7 分钟

How Uber connects its AI agents to models and tools

In this blog we will see how Uber puts two gateways around its AI agents, and how every call carries the person who asked.

aiengg.dev Gaurav Sen
#clippings#AI#agentic#MCP#mcp-gateway#ai-gateway#api-gateway#agent-identity#actor-chain#OAuth#guardrails#control-plane#data-plane#enterprise-ai#Uber

📝 摘要

Uber 于 2025 年初建成供工程师开发 Agent 的平台，如今数以千计的 Agent 运行在其内部服务之上。文章拆解了 Uber 的解法：在 Agent 与模型、Agent 与内部服务之间各设一道网关。所有模型调用经 AI Gateway，由其内置的 AI Guard 做提示注入/越狱检测、不安全内容检查与个人数据脱敏，任何 Agent 都不能直连模型厂商。所有工具调用经 MCP Gateway，把 Uber 服务原生的 HTTP/gRPC/TChannel 接口翻译成 MCP，供 Cursor、Claude Code 和内部 Agent 统一调用。工具由 AutoCrawler 按固定计划爬取服务 API 定义、由 LLM 撰写描述后注册，默认关闭、由服务归属团队审核开启——API 契约变更无需重启网关，只在下次爬取时热更新内存中的工具。文章后半部分解决"Agent 代表谁"的问题：Agent 先从 SPIRE 取得 SVID 证明机器身份，再向 Security Token Service 换取 token，STS 校验 Agent Registry；每个 token 只对一个 audience 有效、仅存活几分钟，并携带完整的 actor chain。文末以七步请求流程串联全链路，给出 token 交换 P99 < 40ms、MCP Gateway 托管 800+ MCP servers / 5000+ tools 的规模数据。

核心概念： Agent 基础设施的难点不在模型本身，而在"连接组织"——通过控制面/数据面分离的网关集群与逐跳身份链，把 Agent 安全地接入企业既有的数千个服务。

关键技术要点：

  1. AI Gateway 与 AI Guard — 模型调用单一出口，护栏做注入/越狱检测、内容安全与 PII 脱敏，把出网数据风险收敛到一个点。
  2. MCP Gateway 协议翻译层 — 接收 MCP 调用后路由到目标服务，并双向翻译为 TChannel/HTTP/gRPC，避免每个团队手写 MCP server。
  3. AutoCrawler + MCP Registry — 定时爬取 API 定义、LLM 生成工具描述、注册为默认关闭状态；Registry 是控制面（决定什么可以运行），Muttley sidecar 承载的数据通路是数据面。
  4. Actor Chain 委托身份 — Agent 不能复用用户凭证，token 逐跳签发（SVID → STS → Agent Registry 校验），每跳一个 audience、数分钟过期，并附带 user1 → oncall-agent → investigation-agent 的完整调用人链。
  5. Token 交换性能 — P99 < 40ms，使逐跳授权在延迟上可接受；被盗 token 只能打开一扇门且仅几分钟。

关键结论： Uber 工程师的总结是 "the hardest part isn't the AI. It's building the connective tissue."——平台价值来自网关、注册表和身份链这些"结缔组织"，而非智能本身。

In early 2025, Uber built a platform for its engineers to build agents.[1] Thousands of agents now work across thousands of its services.

Its engineers faced two problems:

  1. Every agent reached models and services in its own way.
  2. A service couldn't tell which person an agent was acting for.

In this blog we will see how Uber puts two gateways around its AI agents, and how every call carries the person who asked.

1. API Gateways

Some kind of authorization or authentication is common across services. So is rate limiting and request validation.

The logic that can be extracted out and put into a single entry service is put in an API gateway.

A server that stands between the clients and the services, and does their common work, is called an API gateway.

A phone and two desktops send their requests to an API gateway, which sends them on to two services, Payments and Posts, each with its own database. Then the gateway checks every client: authorisation.

2. What does an agent call?

What would a gateway for agents stand in front of? Follow one agent at Uber.

If an on-call engineer starts a session with the Oncall Agent, the Oncall Agent hands part of the work to an Investigation Agent.[1]

Each agent calls two kinds of things. It calls a model to think, and it calls tools to look at Uber's systems.

Many of the models sit outside Uber. OpenAI and Anthropic host some of them. The tools sit inside, in front of thousands of services.

So Uber puts a gateway on each side.

The AI Agent Mesh holds the Oncall, Investigation and Monitoring agents. Their LLM calls go through the AI Gateway to the LLM. Their MCP calls go through the MCP Gateway to the systems behind it.

3. The first gateway: models

What leaves Uber when an agent calls a model? Its prompt, and the prompt may hold a customer's data.

Every call from an agent to a model goes through Uber's AI Gateway.[1] Before the call leaves Uber, the gateway's guardrails, called AI Guard, do three things:

  1. They check for prompt injection and jailbreaks.
  2. They check for unsafe content.
  3. They redact personal data.

No agent at Uber calls a model vendor directly. A gateway that stands between agents and models is called an AI gateway.

4. The second gateway: tools

Who writes the tools an agent calls?

The Investigation Agent now needs data from one of Uber's services. Somebody has to turn that service's API into a tool.

Uber's services speak HTTP, gRPC or TChannel. Agents speak the Model Context Protocol, or MCP. Asking every team to write an MCP server by hand "would be slow and painful".[2]

So the MCP Gateway does three jobs:

  1. It takes MCP calls from every caller.
  2. It routes each call to the right service.
  3. It translates the call into that service's protocol, and the answer back.

Four callers, Cursor, Claude Code, Uber's internal agents and any MCP client, send MCP calls to the MCP Gateway. The gateway calls each service in its own protocol: TChannel, HTTP, gRPC or MCP.

Where do the gateway's tools come from? A crawler called AutoCrawler builds them:[2]

  1. It reads every service's API definition, on a fixed schedule.
  2. An LLM writes each tool's description.
  3. The tool is registered, switched off.
  4. The team that owns the service reviews it and turns it on.

Why on a schedule? Because a service's API changes, and the gateway has to keep up.

The API is literally a contract. So if you send a request, you get a response.

But what if a new type of request has to be added? What if a new API has to be added? What if the API has changed?

So in these cases, there is a change in contract. And therefore, the API gateway needs to be informed of that. And it needs to reflect those changes.

So how do you do that? Restart the API gateway.

And gateway engineers got crazy. They used to spend like 20 to 25% of their sprint times just maintaining the service or just restarting things.

Uber's gateway restarts for none of it. The crawler picks up a changed contract on its next run, and the gateway refreshes its tools in memory.

The part of a system that decides what may run is called the control plane. At Uber, that is the MCP Registry, where teams turn tools on.

The part that carries the calls is the data plane. It sends each call out through Muttley, the sidecar beside every Uber service.

5. Who is the agent acting for?

Would you answer a call without knowing who is asking?

The service now gets a call from the MCP Gateway. It has to decide whether to answer.

Okay, the benefit, like we said, is number one, when requests come in here, you can go for authorization.

Is this user, is Gaurav allowed to make this request to the payment service?

With agents, that question has more than one name in it. The engineer asked the Oncall Agent, which asked the Investigation Agent, which asked for the data.

Without that chain, the service sees only that "some service called an API".[1] The Oncall Agent can't reuse the engineer's own credentials either. The web solved this problem for people a long time ago.

This process of outsourcing user authentication (technically authorization, since the user authorized you to view their name) is called OAuth.

Uber gives its agents the same kind of signed pass. An agent gets one in four steps:[1]

  1. It fetches an SVID from SPIRE. The SVID proves which machine the agent runs on, but not which agent it is.
  2. It asks the Security Token Service for a token.
  3. The token service checks that the agent is registered in the Agent Registry.
  4. The token service returns the token.

An agent gets its token in four numbered steps: it fetches an SVID from SPIRE, asks the Security Token Service for a token, the token service checks the Agent Registry, and the token comes back.

Each token is good for one hop, to one audience, for a few minutes. It carries the names of everyone who asked, in order. That list is called the actor chain.

Uber's post calls the engineer user1. Here is a sketch of the token the Investigation Agent sends to the MCP Gateway. A shared key stands in for the token service's signature.

python
import hashlib
import hmac
import json
import time

KEY = b"held-by-the-token-service-only"

def mint(chain, audience, minutes=5):
    claims = json.dumps({"chain": chain, "aud": audience, "exp": time.time() + 60 * minutes})
    return claims, hmac.new(KEY, claims.encode(), hashlib.sha256).hexdigest()

def accepts(token, me):
    claims, signature = token
    signed = hmac.compare_digest(signature, hmac.new(KEY, claims.encode(), hashlib.sha256).hexdigest())
    body = json.loads(claims)
    return signed and body["aud"] == me and body["exp"] > time.time()

token = mint(["user1", "oncall-agent", "investigation-agent"], audience="mcp-gateway")
print(json.loads(token[0])["chain"])
print("the MCP Gateway accepts it:", accepts(token, "mcp-gateway"))
print("another service accepts it:", accepts(token, "another-service"))
plaintext
['user1', 'oncall-agent', 'investigation-agent']
the MCP Gateway accepts it: True
another service accepts it: False

The token works only where it was sent. A stolen token opens one door, and only for a few minutes.

Final Request Flow

How does the engineer's request reach the service? Here is the whole trip, in seven steps.

  1. The engineer starts a session with the Oncall Agent.
  2. The Oncall Agent calls a model through the AI Gateway.
  3. It gets a token from the Security Token Service, which checks the Agent Registry.
  4. It hands the work to the Investigation Agent, with the chain user1, Oncall Agent.
  5. The Investigation Agent gets a token for the MCP Gateway, with the chain user1, Oncall Agent, Investigation Agent.[1]
  6. The MCP Gateway checks the token and the tool's policy, and redacts sensitive data.
  7. The MCP Gateway calls the service in its own protocol, through Muttley.[2]

A token exchange takes under 40 milliseconds at the 99th percentile.[1] The MCP Gateway now hosts over 800 MCP servers and over 5,000 tools.[2]

Those 5,000 tools are plumbing, and so is every step above. Uber's engineers put the lesson in one line: "the hardest part isn't the AI. It's building the connective tissue".[2]

The whole system, with the request's seven steps numbered: the engineer to the Oncall Agent, its model call through the AI Gateway, its token from the Security Token Service, the handoff to the Investigation Agent, that agent's token, its call to the MCP Gateway, and the gateway's call to the downstream systems.

Summary

  1. An API gateway does the common work of many services, like checking who is calling.
  2. An Uber agent calls models outside Uber and tools inside it.
  3. The AI Gateway carries every model call, with guardrails and personal data redacted.
  4. The MCP Gateway turns every service's API into a tool, on a schedule, with no restarts.
  5. A new tool starts switched off, and the owning team turns it on.
  6. A token service mints one token for each hop: one audience, a few minutes, and the actor chain.
  7. The MCP Gateway hosts over 800 MCP servers and over 5,000 tools.

Sources

  1. Solving the Identity Crisis for AI Agents, Uber Engineering, 21 May 2026.
  2. Designing MCP Gateway: Uber's MCP Management Platform, Uber Engineering, 1 October 2026.
返回 Clippings