MemPalace — a memory palace for agents

What it is

MemPalace is an open-source, local-first long-term memory system for LLM agents. It dresses an otherwise conventional vector-search stack in the cognitive-science metaphor of the method of loci — the ancient "memory palace" mnemonic — and pairs it with one strong, contrarian design choice: store everything verbatim, never summarize at write time. It shipped in April 2026, picked up ~47K GitHub stars in two weeks, and claims state-of-the-art retrieval on the LongMemEval benchmark (96.6% Recall@5) using zero LLM calls on the write path (GitHub repo; recca0120 walkthrough).

The headline pitch is "170 tokens to recall everything": at session start an agent loads only a tiny identity + critical-facts preamble, and pulls deeper memory on demand. Core dependencies are minimal — ChromaDB and PyYAML — and "nothing leaves your machine unless you opt in" (repo README). It exposes itself three ways: a Python API (import mempalace), a CLI (init / mine / search / wake-up / sweep), and an MCP server (python -m mempalace.mcp_server) with ~19–33 tools depending on version, which is how it plugs into Claude Code, Codex CLI, and Cursor.

Why it matters

Two reasons, one real and one mostly marketing.

The real reason: MemPalace is a clean, well-benchmarked argument against the prevailing extract-and-summarize orthodoxy for agent memory. Systems like Mem0 and (in a different way) [[agent-memory-systems]] run an LLM at write time to distill conversations into facts. That costs money, adds latency, is non-deterministic, and — crucially — loses information you didn't know you'd need. MemPalace's counter-bet is that embeddings are now good enough that you can just store the raw text, embed it, and let semantic search do the recall. The benchmark numbers say the bet largely pays off: a fully deterministic, offline, zero-API write path that still tops the table (arXiv 2604.21284).

The marketing reason: the palace metaphor (Wings → Rooms → Drawers) is presented as a retrieval innovation. A 2026 critical-analysis paper ("Spatial Metaphors for LLM Memory", arXiv:2604.21284) shows it is not — but it is a genuinely good ergonomic for humans reasoning about what an agent remembers, which has its own value (see Honest caveats).

For OpenAlice specifically it matters because OpenAlice already committed to a file-first, no-LLM-at- write-time memory architecture (see the global memory note feedback-storage-file-first-simplified). MemPalace is the closest well-documented external prior art to that decision, with hard ablations we can borrow.

How it works

The spatial data model (method of loci)

MemPalace organizes memory as a hierarchy that maps onto the loci mnemonic (recca0120 walkthrough):

• Wing — a top-level organizing unit: a person, a project, a major topic.
• Room — a sub-topic inside a wing (e.g. auth, billing, deploy).
• Hall — cross-wing "corridors" by memory type, shared across all wings: hall_facts (locked decisions), hall_events (sessions/milestones), hall_discoveries, hall_preferences, hall_advice.
• Closet — compressed AAAK summaries that point to the originals (the index layer).
• Drawer — the verbatim original content, preserved losslessly. This is the source of truth.
• Tunnel — a cross-wing link connecting duplicate/related rooms.

Mechanically, all of this is metadata on documents in a vector store. A "wing"/"room" is a tag; "search within a wing" is a metadata-filtered similarity query. That's the whole trick — and it's why the critique paper calls the hierarchy "metadata filtering — an effective but well-established technique."

The write path (deterministic, no LLM)

Ingestion happens via mine (bulk: code repos, conversation transcripts, Slack exports) and sweep (per-message verbatim drawer storage). For each unit of content the pipeline is, in prose:

1. Take the raw text → write it to a drawer (verbatim, lossless).
2. Compute an embedding (default embedding-gemma-300m, multilingual; or all-MiniLM-L6-v2, 384-dim English-only, ~30 MB).
3. Optionally produce an AAAK closet summary and link it to the drawer.
4. Insert into the backend with wing/room/hall metadata.

No LLM is invoked anywhere in that path — embeddings are a local model, AAAK is a deterministic string transform. That is the defining property: writes are free and offline.

AAAK compression and the four memory layers

AAAK is a deterministic "AI-readable shorthand" — structured abbreviation that any LLM can read natively without a decoder. Example from the walkthrough: a ~1,000-token team description collapses to TEAM: PRI(lead) | KAI(backend,3yr) SOR(frontend) ... DECISION: KAI.rec:clerk>auth0(pricing+dx) at roughly 120 tokens. Marketing says "30x compression, zero information loss"; the ablation says realistic compression is ~8–10x and it is lossy (you cannot reconstruct the original — that's what the drawer is for).

Memory is loaded in four layers, progressively:

wake-up emits L0+L1 only — the "~170 tokens to recall everything" claim. L2/L3 are deferred until the conversation actually needs them, so the steady-state context cost is tiny.

The read path: three retrieval modes

1. Raw — pure embedding similarity over verbatim drawers, no heuristics, no LLM. 96.6% R@5.
2. Hybrid v4 — adds keyword boosting, temporal-proximity boosting, and preference-pattern extraction on top of raw. ~98.4% held-out.
3. LLM rerank — take top-20 candidates and re-rank with a model-agnostic LLM reader. ≥99%.

Scoping the search by palace structure measurably raises recall — but this is metadata filtering, not a new algorithm:

Pluggable backends and the temporal knowledge graph

Retrieval is pluggable behind a single interface in mempalace/backends/base.py. Defaults to ChromaDB (local); also supports SQLite_exact (correctness oracle), Qdrant, and pgvector.

Separately, there's a temporal entity-relationship graph in local SQLite where every fact carries a validity window:

kg.add_triple("Kai", "works_on", "Orion", valid_from="2025-06-01")
kg.invalidate("Kai", "works_on", "Orion", ended="2026-03-01")
kg.query_entity("Kai", as_of="2026-01-20")

Invalidation marks an end date rather than deleting — so "as-of" queries return what was true at a point in time and stale facts don't corrupt decisions. This is real and useful, though limited (see caveats): it's a flat triple store, single-hop only.

Agents, diaries, and auto-save hooks

Each specialist agent gets its own isolated vault under ~/.mempalace/agents/<name>.json, its own wings, and an AAAK diary written via mempalace_diary_write() / read via mempalace_diary_read(last_n). Agents are discoverable at runtime (mempalace_list_agents). Two hooks keep memory fresh without manual effort: a save hook firing every ~15 messages (auto-extracts topics/decisions/code changes) and a PreCompact hook that emergency-saves before the host's context window is compacted.

Key ideas & tradeoffs

• Verbatim-first vs extract-first. The central thesis. Verbatim never loses information and needs no LLM to write, but it stores more bytes and leans entirely on embedding quality at read time. Extract-first (Mem0, [[agent-memory-systems]]) is denser and pre-resolves contradictions but is lossy, costly, and non-deterministic. MemPalace bets retrieval > distillation.
• Deterministic, offline, zero-cost writes. Genuinely distinctive. Cost comparison from the docs: paste-everything ≈19.5M tokens (impossible), LLM-summaries ≈$507/yr, MemPalace wake-up ≈$0.70/yr, wake-up + 5 searches ≈$10/yr.
• Progressive layering (L0–L3). Pay context tokens only for what the turn needs. This is the part worth stealing regardless of whether you like the palace metaphor.
• Metadata-scoped search as a recall multiplier. Not novel, but a real, free +34% on this benchmark — the same lesson [[graphrag]] teaches from the graph side: constrain the candidate set before you rank.
• LLM rerank as an optional top layer. Mirrors the pattern in [[model-routing]] and [[fusion-and-llm-councils]] — cheap deterministic recall first, expensive model only on the short list.

Honest caveats & limitations

The critical-analysis paper (arXiv:2604.21284) is unusually direct, and the project's own benchmarks were walked back; both are worth taking at face value.

• The spatial metaphor does not improve retrieval. Verdict, quoted: "the spatial metaphor does not improve retrieval accuracy, reduce latency, or enable capabilities that a flat vector database cannot provide." The +34% comes from metadata filtering, "documented in every vector database's getting-started tutorial." The palace is a human ergonomic, not an algorithmic one.
• Performance is mostly the embedding model + verbatim storage, not MemPalace per se. Swap in the same embeddings and verbatim policy on a bare ChromaDB and you reproduce most of the 96.6%.
• AAAK is lossy, ~8–10x not 30x. The "30x, zero information loss" claim is false; the measured hit is a 12.4-point Recall@5 drop (96.6% → 84.2%) when retrieving from AAAK instead of drawers. AAAK is fine as an index/preamble, not as the retrieval substrate.
• Benchmark overclaiming, since corrected. Early "100% LongMemEval" required iterative LLM reranking plus test-fix cycles; "100% LoCoMo" used top_k=50 (i.e. retrieve the entire conversation). "Contradiction detection" was exact-match dedup, not semantic.
• Knowledge graph is shallow. A flat triple store with single-hop lookups and no entity resolution — contrast Zep's Neo4j multi-hop graph. No spreading activation, so none of the associative recall that the loci metaphor implies.
• Scalability ceiling. Single ChromaDB collection; expect trouble past ~1M+ documents.
• The moat is thin. Mem0's April-2026 token-efficient update narrowed the gap from ~49% to 93.4%, so the verbatim advantage is real but not permanent.

Net: MemPalace is "significant architectural insight wrapped in overstated claims." The insight — verbatim storage + good embeddings + zero-cost deterministic writes + progressive layering — is sound and reusable. The palace branding is the part to be skeptical of.

How it connects to OpenAlice

OpenAlice's memory architecture is file-first by NAO directive (feedback-storage-file-first-simplified-2026-05-20): memory lives as plain files on disk, written without an LLM in the hot path, with per-chat scoping (e.g. memory/<workspace>/chat/<channel>/...). MemPalace is the strongest external validation of that decision, plus a few concrete things to take and a few to avoid:

• Verbatim drawers ≈ OpenAlice's raw message files. Both refuse to summarize at write time and keep the original as source of truth. MemPalace's ablation gives us a number to defend that: extract-first (AAAK-only) loses 12.4 points of recall. Keep storing raw.
• The L0/L1 "~170-token wake-up" maps directly onto OpenAlice's per-turn prompt-composer budget. OpenAlice already cares about byte-exact, minimal preambles (feedback-prompt-byte-identical- preservation). A deterministic L0(identity)+L1(critical-facts) preamble, with L2/L3 pulled on demand, is a clean way to bound context cost — worth prototyping in the prompt composer / wake-up analog.
• Metadata-scoped retrieval is the free win to copy. OpenAlice's per-chat directory structure is already a wing/room metadata scheme on disk. The lesson: always constrain recall to the chat/topic scope before ranking — same principle [[graphrag]] and our own [[graphify]] AST scoping exploit.
• Steal the validity-window KG idea, but go deeper than they did. A temporal triple store with valid_from/invalidated and as_of queries would let Alice answer "what was true then" without stale-fact corruption — but build multi-hop + entity resolution from the start, since MemPalace's flat single-hop graph is its weakest part. (Atlas itself is a multi-hop code/knowledge graph; the pattern is in-house.)
• MCP surface parallels our tooling. MemPalace exposes memory as MCP tools (diaries, search, KG ops); OpenAlice already speaks MCP across the lab (Atlas a2a, telegram, etc.), so a memory MCP server is a natural shape if we ever externalize Alice's memory for sub-agents.
• Don't import the palace metaphor as architecture. Per the critique, Wings/Rooms/Drawers buy nothing over our existing file-path scoping. Keep the file-first model; borrow only the mechanics (verbatim + deterministic writes + layered loading + scoped retrieval + temporal KG).

For routing the optional LLM-rerank tier, see [[model-routing]] and the councils pattern in [[fusion-and-llm-councils]]; for the graph-scoped retrieval analogy see [[graphrag]] and [[graphify]]; for the broader design space of agent memory (Zettelkasten/A-MEM dynamic linking, MemGPT tiering) see [[agent-memory-systems]]; for the self-maintaining-knowledge pattern this library uses see [[llm-maintained-wiki]]. Builders coming from first principles can ground the embedding/retrieval parts via [[llm-from-scratch]] / [[microgpt-build-an-llm-from-scratch]], and the multi-reader rerank tier via [[mixture-of-agents]]. ([[mempalace]] is this page.)

Contrast worth keeping in mind: A-MEM

A-MEM (arXiv:2502.12110) takes the opposite stance to MemPalace: a Zettelkasten-style network where notes dynamically link and evolve — new observations trigger synthesis with existing notes, high-connectivity nodes surface organically, and detail compresses upward into principles. Where MemPalace says "archive raw, let embeddings retrieve," A-MEM says "memory must be structured, relational, and continuously refined." OpenAlice sits closer to MemPalace today (file-first, no write-time LLM), but A-MEM is the design to watch if we ever want associative recall rather than pure similarity search.