Building Self-Correcting Memory in OpenWiki
Memory Has a Forgetting ProblemA memory system is only as good as its ability to forget.Most discussions about agent memory focus on creatin...
📝 摘要
本文介绍 OpenWiki 0.4.0 的自修正记忆系统（Self-Correcting Memory），核心解决 Agent 长期记忆中"遗忘"问题——当源代码（source of truth）随时间演变时，Wiki 中曾经正确的事实会变得过时（stale），导致记忆漂移（memory drift）和记忆投毒（memory poisoning）。
OpenWiki 引入 Claims Runtime，将 Wiki 页面中的每条事实声明（claim）与代码中的支撑证据（evidence）建立版本化链接。当源代码变更导致证据版本不同步时，该 claim 被标记为 stale，不确定性持久保留直到下次更新时重新验证。验证后，若 claim 仍成立则刷新证据版本，若已错误则同步修正 Wiki 内容和证据。这形成了自我修正的反馈循环，且更新成本随代码变更量而非声明总量扩展。
文章还介绍了 OKF (Open Knowledge Format) v0.2，将 claim 证据、生产者信息和验证状态投射到可移植的 Markdown 元数据中，实现 Wiki 页面在 OpenWiki 外部的可检查性。评估显示，Claims Runtime 将 stale claims 从 80 降至 9，hallucinated claims 从 15 降至 0，且在单次运行中 stale claims 从 17% 回落至 0%。
核心概念： 记忆系统的自修正能力来自"声明-证据"的版本化绑定——不只记住自己相信什么，还记住为什么相信，当证据变化时标记不确定，直到重新验证或修正。
关键技术栈 / 关键要点：
- Claims Runtime — 将 Wiki 声明与代码证据版本化绑定，通过版本比对确定性检测 staleness，无需模型调用
- Staleness ≠ Wrongness — Stale 仅表示"不再能安全假设为真"，不确定性持久保留直到证据被重新检查
- Self-Correcting Loop — 代码变更→声明变 stale→下次更新时验证/修正→Wiki 回归同步，更新成本与代码变更量而非声明总量成比例
- OKF v0.2 — 将 provenance/evidence sources/verification history 作为标准信任摘要暴露在 Markdown front matter 中，实现 Wiki 页面的可移植可检查性
- Evaluation Results — Stale claims 从 80→9，Hallucinated 从 15→0；单次运行中 stale 从 17% 回落至 0%，supported 从 77% 恢复至 98%
- Deterministic Staleness Check — 每次 Wiki 更新前全量扫描 claim set 比对证据版本，无模型调用，数千 claims 仍保持快速
关键结论 / 实践建议： 遗忘不是删除旧记忆，而是知道何时不应再信任某个信念。Claims Runtime 使 OpenWiki 能够主动遗忘和随源代码进化，而非逐渐漂移脱离。
Memory Has a Forgetting Problem
A memory system is only as good as its ability to forget.
Most discussions about agent memory focus on creating memories and making them retrievable later. Long term memory introduces another interesting problem that is often overlooked. As the source of truth changes, facts that were once correct can become stale without the memory system having any reliable way to notice. Over time, this causes the memory system to degrade due to things like memory drift and memory poisoning.
This problem is especially obvious in memory systems like OpenWiki because the source of truth is code that evolves over time. At first, a wiki might accurately describe API behavior or various features that exist, but as the code changes those behaviors and features may also change. If the documented knowledge about them doesn’t stay in sync with the code then those parts of the wiki become stale.
To make OpenWiki capable of forgetting and self-correcting, we needed a way to preserve not just what the wiki believes, but the evidence behind those beliefs. That gives OpenWiki a way to notice when the evidence supporting a claim has changed and identify what parts of the wiki may need to be revisited.
Grounding What the Wiki Believes
The first step was to give OpenWiki a persistent record of the material factual claims it makes. This means that when the agent writes a wiki page, it must also identify the claims in that page and the evidence in code that supports them.
As an example, if the wiki says that failed tasks are retried three times by default, OpenWiki records that claim alongside the code that supports it.
The runtime then records a version for the supporting evidence so the claim can be validated again later. Doing this creates a link between each claim in the wiki and the code that supports it. Instead of treating the wiki as a collection of text, OpenWiki can track individual claims and the evidence they depend on.
Knowing When Knowledge Goes Stale
Once a claim is linked to evidence, OpenWiki can detect when that evidence changes by comparing the version of the source that originally supported a claim against the version that exists. If they differ, the claim is flagged as stale.
A stale claim does not necessarily mean that the claim is wrong. It means that OpenWiki can no longer safely assume that the claim is still true without checking the source again. Many code changes do not immediately invalidate a dependent claim, but they do create a reason to reconsider it.
OpenWiki does not need to persist a separate status flag for this. The stored evidence version is enough to validate the claim’s freshness again on every update, so uncertainty remains durable until the evidence is rechecked.
Correcting Stale Claims
Detecting stale knowledge is only useful if the system has a way to correct it.
When OpenWiki determines that a claim is stale, it can inspect the current supporting evidence and determine if the claim is true or false. If the claim remains true, OpenWiki can refresh the evidence version. If the claim is now false, the wiki content and its supporting evidence are updated together.
The important behavior is that stale claims remain stale until that verification actually takes place. Over time, this creates a self-correcting feedback loop. Changes to the source make previously trusted claims stale and that uncertainty persists until each claim is validated again. Wiki updates bring the documentation back in sync with what is true now.
This is different from just regenerating the wiki from scratch. OpenWiki persists what it already knows, tracks which claims have become stale, and corrects those claims as the source continues to evolve.
Connecting Claims to OKF v0.2
Claims give OpenWiki an internal record of what each page believes and the versioned evidence that supports it. Open Knowledge Format (OKF) v0.2 makes the page level result portable and inspectable outside of OpenWiki.
OpenWiki projects claim evidence into each page's OKF sources metadata, records which producer last changed the page body with generated, and adds verified only after the page's complete claim set has been reconciled, rechecked, and persisted. The wiki's root index declares okf_version: "0.2", making the Markdown bundle recognizable to other tools that consume OKF.
The creates a useful separation of responsibilities. Claims determine whether individual beliefs remain trustworthy. OKF communicates the provenance, evidence sources, and verification history of the document containing them. Detailed claims and precise evidence versions stay in OpenWiki’s sidecars, while the portable Markdown exposes a standard trust summary as part of the OKF front matter.
How This Runs in Practice
Staleness detection runs at the start of every wiki update, before the agent does anything. The runtime walks the full claim set and compares each claim's persisted evidence version against the current source. This is a deterministic check with no model calls, so it stays fast even as the wiki accumulates thousands of claims.
The agent never sweeps the claim set itself. When it reads a page during an update, any stale claims on that page are surfaced alongside the content, and it resolves them as part of the work it is already doing: re-verify the claim and refresh its evidence, or correct the claim and the wiki text together. Anything left unresolved stays flagged for a future update rather than being silently dropped.
The practical effect is that update cost scales with how much the code changed, not with how many claims the wiki holds.
Evaluating the Ability to Forget
To measure this, we built an evaluation that replays a code repository through a series of git commits with pre-defined checkpoints where the wiki is updated. The various commits introduce new features, changes to existing behavior, bug fixes, and reverts.
After each update, we evaluated factual claims in the wiki and classified each as being one of the following:
- Supported: the claims is backed by the code at the current checkpoint.
- Stale: the claim was accurate an earlier checkpoint, but the code has since changed and it no longer holds.
- Hallucinated: the claim was never true at any point in the repo’s history.
- Unverified: the claim can’t be confirmed or refuted against the code.
We ran the same sequence with and without the OpenWiki claims runtime.
Across each update we saw that stale claims fell from 80 in the baseline to 9 with the claims runtime, while hallucinated claims fell from 15 to 0.
We also saw the self-correction behavior directly in individual runs. In one example, a code change left 17% of the wiki’s claims stale at the first checkpoint. By the next checkpoint, stale claims had returned to 0% and supported claims had recovered from 77% to 98%
This is the behavior we care most about. The claims runtime gives OpenWiki a way to identify when existing knowledge may no longer be trustworthy, persist that uncertainty across updates, and later correct it as the code continues to evolve.
What Makes Memory Self-Correcting
Memory systems must have a way to say what they believe and why they believe it. More importantly, when something that was once valid becomes stale, the system must have a way to forget and self-correct.
The claims runtime gives OpenWiki that capability. Claims durably connect wiki knowledge to source evidence so that changes to the evidence flag affected claims as stale. Future wiki updates then have the ability to validate or self-correct stale claims without rebuilding the wiki from scratch.
In this model, forgetting is not all about deleting old memories. Instead, it is about knowing when a belief should no longer be trusted. This gives OpenWiki a way to proactively forget and evolve with the source code it represents instead of slowly drifting away from it.
Try OpenWiki 0.4.0
If you already use OpenWiki, upgrading to 0.4.0 will automatically begin generating claims for your wiki and migrate to OKF v0.2 in your next update. Check out the repo or learn more about OpenWiki using our official documentation:
Try it out today:
npm install -g openwiki@latest
openwiki --init