Can your AI remember long conversations - correctly and affordably?
How Condensate compares on the LoCoMo industry memory test - 10 long chats, ~2,000 questions.
Generated 2026-06-10 · locomo10_full_report.json
The problem we are solving
Long-running AI assistants must remember what users said weeks ago - without resending entire chat logs every time.
- Wrong answers when the right fact never reaches the model.
- Stale facts when users correct themselves but old information remains.
- High cost when every question includes tens of thousands of tokens of history.
Our goal - and where we stand today
Production goal: ≥ 85% retrieval accuracy at < 7,000 tokens per question.
Industry reference: ~92.5% overall · ~6,956 tokens/question.
83.6%
Condensate - right answer present in memory
83.6% vs 85% goal (1.4 pts to go). Full transcript: 80.4%.
1,647
Tokens of memory per question
Under 7k budget. ~20,476 for full transcript.
What each benchmark approach means
- Full transcript - paste the whole chat (80.4%, 20,476 tok/q).
- Observation list - extracted fact bullets (69.2%, 6,307 tok/q).
- Structured notes - organized storage, no supersession (80.4%, 22,128 tok/q).
- Industry reference - published leaderboard (~92.5%, ~6,956 tok/q).
- Condensate - updating assertion memory (83.6%, 1,647 tok/q).
Strengths and gaps
- Cost: ~12× fewer tokens than full transcript at similar recall.
- Open-domain: 92.5% vs industry 76.0% ✓.
- Temporal: 95.6% vs industry 92.8% ✓.
- Memory updates: supersession + provenance (see ContradictionBench).
| Type | Industry ref. | Condensate | Full transcript |
|---|
| Open-domain | 76.0% | 92.5% | 98.3% |
| Temporal | 92.8% | 95.6% | 86.0% |
| Single-hop | 92.3% | 84.0% | 92.9% |
| Multi-hop | 93.3% | 81.2% | 55.2% |
| Adversarial | - | 58.3% | 40.1% |
Gaps: multi-hop (81.2%), adversarial (58.3%), overall +1.4 pts to 85% goal.
Why this drives adoption
- Lower LLM bills for always-on agents.
- Trust when user facts change over time.
- Strong on broad and time-based recall vs industry reference.
- Transparent fair testing (per-conversation ingest, session-scoped).
Bottom line: Credible cost-efficient memory on a hard public benchmark - close to production accuracy target, leading on price-performance and several categories.