trust
Do not take it on faith. Read the numbers.
Every figure on this page carries its denominator, the grader that produced it and the caveat that limits it.
open benchmark
One open conversation set, three graders, the same answers.
LoCoMo is a public long-conversation set. It measures retrieval: whether the right memory comes back. It does not measure the three mechanics only recall has.
The production run covers all ten LoCoMo conversations, categories one to four, which are the fact-recall questions. That is 1,540 scored questions.
Every answer came from memory on the production service rather than from the raw transcript. Three rubrics then scored the same answers side by side: our own strict grader, a Zep-style grader and a Mem0-style grader.
| Grader rubric | Production run | Previous engine | n |
|---|---|---|---|
| strict (ours) | 81.5% | 70.8% | 1,540 |
| Zep-style | 90.2% | 82.5% | 1,540 |
| Mem0-style | 92.8% | 90.1% | 1,540 |
The answerer and judge models differed between the two runs. So the claim is no regression, not that the new engine improved the score. The previous engine run used gpt-5-mini to answer and gpt-4o-mini to judge.
No full-context baseline was run at this scale, so these numbers say nothing about memory against handing the model the whole transcript.
Adversarial questions, from the previous engine run
Category five is the trap questions, where the job is to refuse rather than to answer. These shares come from the previous engine run of 2026-09-07 and are not part of the 1,540 above.
| Adversarial (cat 5) | Share | n |
|---|---|---|
| Fell for the trap | 33.6% | 446 |
| Abstained | 49.6% | 446 |
Excluded from the fact-recall total on purpose, because the target answer there is a refusal. The file reports only these two shares for the category.
runs
- production
LoCoMo on the production service
Ten conversations, 1,540 scored questions, three rubrics over one set of answers: 81.5% strict, 90.2% Zep-style, 92.8% Mem0-style.
locomo-ten-gpt5mini-improved.json
The previous engine baseline. Ten conversations, 1,540 fact-recall questions: 70.8% strict, 82.5% Zep-style, 90.1% Mem0-style, plus the adversarial shares above.
every LoCoMo number above comes from one of those two runs.
Not measured yet: the repeat rate.
The counted test is how often an agent repeats a mistake a person already corrected, with a CLAUDE.md file, with claude-mem and with recall, on the same set of corrected mistakes. It has not run. When it does, it goes here with its denominator.
judgment run
Thirty decisions, replayed with and without the memory that preceded them.
The question is narrow: given only what memory held before a decision was made, does the model choose what the owner chose?
Thirty real logged decisions were replayed in three conditions. Run 2026-09-06, judged by DELTA, a local model, on an idle gateway.
Bare is no memory at all. Before is memory as it stood ahead of the decision. After is memory with the decision itself already in it.
| Condition | Same choice | Reason raised |
|---|---|---|
| bare | 66.7% (20/30) | 70.0% |
| before | 70.0% (21/30) | 70.0% |
| after | 100% (30/30) | 96.7% |
The lift from bare to before is 3.3 points, which is roughly one additional matching decision in thirty. Thirty cases is a small sample and a pilot metric, not a claim about judgment in general.
The 100% in the after row is a recall check, not a judgment result. The decision is already in memory there, so it must never be read as if it were the before number.
data path
Where the text goes, and where it does not.
Redaction runs before storage. Extraction runs on the endpoint you configure, including a local one. There is no vendor of ours on this path.
These shapes are replaced before a slice, a source or a record is stored, by the redaction pass rather than by a person remembering to.
- Key shapes:
sk-,AKIA,ghp_,xox,AIza, JWTs and private keys. - Labelled secrets:
password:,token=,api_key:. - Credentials carried inside connection strings.
- Luhn-valid card numbers.
- Your own switch on top of that:
<private>tags and<recall-off>.Text in private tags never leaves the machine, and a session containing<recall-off>is never shipped at all.
chat and extraction run on the endpoint you set.
nothing silently overwritten
A record is superseded, not overwritten.
History is the point. If a record could quietly change, no brief built on it could be trusted. A record is forgotten only when a person asks.
Superseded, never overwritten
A newer record supersedes an older one instead of replacing it. Both stay readable, and the outcome on the old one stays as it was.
Decisions carry an outcome of held, reversed or regret, set with one click. Reversed ones are marked in every brief after.
History and chains kept
Every record page shows its history and its chain: what it supersedes, what it led to, what precedes it, what contradicts it.
Chains is a timeline you can scrub, so a decision can be read in the order things actually happened.
Every action in Audit
Audit lists actions with the actor who took them, filterable by actor, by action and by date range.
Records also say who taught them, and whether the words were a person's or an agent's reply.
Yours to take out
Any space exports as Markdown or JSON from its own page, so the record set is never only inside the app.
Forgetting happens only when a person asks. Today that request scrubs the content, keeps the title and links as a tombstone, and is itself audited.
early access
Connect, once you have access.
Early access is by invitation while we load-test capacity and cost. With access, every MCP client uses one endpoint.
https://app.recall.hiretrainva.com/mcp