Agent Memory Architecture: The Four Tiers Behind a Stateful AI Agent

A large language model, on its own, remembers nothing. Every call into it is a cold start: you hand it a block of text, it produces a block of text, and the moment the response is returned the model has no idea any of it happened. That is a perfectly reasonable design for a text transformer. It is a terrible design for an agent.

An agent is supposed to be the opposite of stateless. It should know who you are, remember that it already tried and failed a particular approach twenty minutes ago, recall that this exact production incident happened last quarter and how it was resolved, and stay inside the behavioural rules you set for it. None of that comes from the model weights. All of it comes from memory architecture — the scaffolding you build around the model to make a stateless function behave like a stateful, self-improving system.

The centre of the diagram is deliberate. Working memory sits in the middle because it is the only tier the model actually sees. Everything else in the picture exists to decide what gets loaded into that centre box, and what gets written back out of it.

Follow the arrows and you get the agent’s lifecycle:

• Procedural memory flows down into working memory as behavioural guidance — the rules, the tool schemas, the system prompt.
• Semantic memory flows in from the left — you issue a semantic query, and user profiles and durable facts come back.
• Episodic memory flows in from the right — a vector search over past execution traces returns relevant previous experiences and the solutions that worked.
• Tools hang off the bottom — working memory dispatches tool calls, and observations come back into the message stream.
• Consolidation flows back outward — the orange return paths. Extracted facts and preferences are written into semantic memory; the finished execution trace is written into episodic memory.

That last set of arrows is the part most implementations skip, and it is the part that makes the difference between an agent with memory and an agent with a really long prompt. Retrieval without consolidation means the agent can only ever know what you manually put into the store. Consolidation is what lets it learn.

The four tiers, and what each one is actually for

The taxonomy borrows from cognitive psychology, but the engineering distinctions are real: each tier has a different lifespan, a different access pattern, and a different piece of infrastructure behind it.

1. Working memory — the in-context state

This is the agent’s scratchpad for a single run. It holds the message stream you can see in the diagram: the user prompt, the agent’s own thoughts, the tool calls it emitted, and the observations that came back. In LangGraph this is literally the graph’s state object, and it is governed by two hard constraints — the model’s context limit, and whatever state reducer you attached to each field.

The important property is that it is checkpointed at super-step boundaries. After every node completes, the full state is serialised. That gives you three things almost for free: you can inspect exactly what the agent believed at any step, you can roll back to a previous checkpoint for human-in-the-loop correction, and you can build recovery loops that resume a failed run instead of restarting it.

2. Episodic memory — experience trajectories

Episodic memory stores what happened, structured as trajectories: problem → action → observation → solution. This is the tier that lets an agent say “I have seen this before.”

The access pattern is similarity search, not lookup. You do not know the key of the relevant past incident — that is the whole point. You take the user’s current query, embed it, and search for the episodes whose stored content lands nearby in vector space. A user reporting “the batch job keeps dying halfway through” should surface last month’s out-of-memory incident even though not a single keyword matches.

3. Semantic memory — knowledge and profiles

Semantic memory stores what is true, stripped of the context in which it was learned. The user prefers AWS. Their primary stack is Python and FastAPI. They are a principal engineer. Their team’s deployment window is Tuesday mornings.

The distinction from episodic memory is worth being precise about, because it drives the retrieval strategy. An episode is a dated event you search by similarity. A semantic fact is a timeless assertion you fetch by key. You almost always want the user’s profile on every turn, so fetching it directly by a known key is both cheaper and more reliable than hoping a vector search surfaces it.

4. Procedural memory — behaviour and heuristics

Procedural memory is the tier engineers tend not to think of as memory at all, because it lives in code rather than in a database. But it fits the definition: it is knowledge the agent retains across every execution, and it shapes behaviour without being re-derived each time.

It includes the hardcoded graph topology (which node can follow which), the JSON schema contracts that validate tool arguments, the constraints on when a tool may fire, and the behavioural policies embedded in the system instructions. In the diagram it is drawn at the top, feeding downward, because it is the tier that constrains all the others.

How the data actually flows

Put the tiers in motion and you get a cycle, with working memory acting as the runtime execution kernel:

1. Ingress and pre-fetch. A user invokes the agent. Procedural guidelines format the system prompt. Working memory issues a semantic key-value lookup for the profile and a vector search across the episodic namespace.
2. Reasoning and tool execution. The model processes the assembled prompt. If information is missing or an action is required, it dispatches tool calls. Results — and errors — are folded back into working memory as observations.
3. Memory consolidation. As tasks resolve, dedicated consolidation steps (or tools the agent calls itself) write new facts into semantic memory and finished execution episodes into episodic memory.

The distinction that trips everyone up: checkpointer vs. store

If you take one implementation detail away from this post, make it this one. LangGraph gives you two different persistence mechanisms, and they are not interchangeable.

A checkpointer (MemorySaver, PostgresSaver) is thread-isolated runtime state persistence. It serialises the full message history for one thread_id. It is how a conversation survives between turns. It is not how anything survives between conversations — open a new thread and the checkpointer gives you nothing.

A BaseStore (InMemoryStore, PostgresStore) is global, cross-thread hierarchical storage, organised by (namespace, key) and optionally backed by a vector index. This is where long-term memory actually lives.

Checkpointer for the conversation. Store for the memory. Reach for the wrong one and you will build an agent that either forgets everything the moment a session ends, or leaks one user’s context into another’s.

Implementation: vector-indexed semantic search

Start with the retrieval half. A BaseStore can embed specific fields on insertion, which turns it into a vector index you can run cosine similarity searches against via store.search().

from langchain_openai import OpenAIEmbeddings
from langgraph.store.memory import InMemoryStore

# Configure store with embedding index
embedding_model = OpenAIEmbeddings(model="text-embedding-3-small")
long_term_store = InMemoryStore(
    index={
        "embeddings": embedding_model,
        "dims": 1536,
        "fields": ["content"],  # Automatically embeds the 'content' field
    }
)

# Search across user's episodic or knowledge namespaces
results = long_term_store.search(
    namespace=("users", "user_101", "knowledge"),
    query="database configuration and replication",
    limit=3
)
for item in results:
    print(f"Match score: {item.score:.2f} | Content: {item.value['content']}")

Three parameters in that index block are doing all the work. embeddings names the model. dims must match that model’s output dimensionality — 1536 for text-embedding-3-small — and getting it wrong fails at insertion time. fields is the interesting one: it tells the store which keys inside your stored dictionary to actually embed. Here only content is indexed, so you can attach arbitrary metadata to the same record (timestamps, severity, ticket IDs) without polluting the vector.

Note the namespace tuple. It is hierarchical, and it is your isolation boundary — ("users", "user_101", "knowledge") scopes the search to one user’s knowledge and nothing else.

Implementation: all four tiers in one LangGraph agent

Now the full build. It integrates all four memory models, on-demand tool calling via InjectedStore, and persistent checkpointing.

Procedural memory as explicit policy

import uuid
from typing import Annotated, TypedDict
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
from langchain_core.runnables import RunnableConfig
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
from langgraph.prebuilt import InjectedStore, ToolNode, tools_condition
from langgraph.store.base import BaseStore
from langgraph.store.memory import InMemoryStore

# --- 4. PROCEDURAL MEMORY (Behavioral Rules & Policies) ---
PROCEDURAL_GUIDELINES = (
    "You are an enterprise AI assistant. Adhere strictly to these procedural rules:\n"
    "1. Always format responses using technical, concise bullet points.\n"
    "2. Consult episodic memory when addressing system crashes or infrastructure issues.\n"
    "3. Adhere strictly to user technical preferences stored in semantic memory."
)

Procedural memory here is just a module-level constant — and that is the point. It is version-controlled, diffable, and reviewable. Rule 2 is quietly the most important line in the block: it is a routing instruction that tells the model when episodic recall is relevant, which is what stops the agent from either ignoring its history or dredging it up on every trivial turn.

Episodic and semantic memory as tools

# --- 2. EPISODIC & 3. SEMANTIC MEMORY TOOLS ---
@tool
def record_resolved_episode(
    problem: str,
    solution: str,
    *,
    config: RunnableConfig,
    store: Annotated[BaseStore, InjectedStore()],
) -> str:
    """Saves a resolved technical incident/task into episodic memory for future recall."""
    user_id = config.get("configurable", {}).get("user_id", "default_user")
    namespace = ("users", user_id, "episodic_memory")
    entry = f"Problem: {problem} | Solution: {solution}"
    store.put(namespace, key=str(uuid.uuid4()), value={"content": entry})
    return f"Successfully saved episodic memory: {problem}"

@tool
def update_user_profile_fact(
    preference_key: str,
    value: str,
    *,
    config: RunnableConfig,
    store: Annotated[BaseStore, InjectedStore()],
) -> str:
    """Updates semantic profile memory (e.g. language preferences, tech stack)."""
    user_id = config.get("configurable", {}).get("user_id", "default_user")
    namespace = ("users", user_id, "semantic_profile")
    current_item = store.get(namespace, "profile")
    profile = current_item.value if current_item else {}
    profile[preference_key] = value
    store.put(namespace, key="profile", value=profile)
    return f"Updated user preference: {preference_key} = {value}"

tools = [record_resolved_episode, update_user_profile_fact]

This is memory consolidation implemented as tool calls — the agent decides when something is worth remembering. Two mechanisms deserve attention.

Annotated[BaseStore, InjectedStore()] is the critical piece. It marks store as a runtime-injected dependency, which means the parameter is hidden from the model’s tool schema. The model sees a tool that takes problem and solution; the framework supplies the store handle. Without this you would be asking an LLM to hallucinate a database connection.

The keyword-only marker * enforces that split at the signature level: everything before it is model-supplied, everything after is runtime-supplied.

Also notice the difference in write strategy between the two tools, which mirrors the difference between the tiers. Episodes are written under a fresh uuid4() — every incident is a new, immutable record, because history is append-only. The profile is written under the fixed key "profile" after a read-modify-write — facts get updated in place, because you want the current truth, not a pile of contradictory snapshots.

Working memory and the store

# --- 1. WORKING / SHORT-TERM MEMORY (State Definition) ---
class AgentState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]

# Vector-indexed store for Episodic and Semantic lookups
long_term_store = InMemoryStore(
    index={
        "embeddings": OpenAIEmbeddings(model="text-embedding-3-small"),
        "dims": 1536,
        "fields": ["content"],
    }
)

# Seed Long-Term Memories
long_term_store.put(
    namespace=("users", "engineer_01", "semantic_profile"),
    key="profile",
    value={"preferred_cloud": "AWS", "primary_stack": "Python/FastAPI", "role": "Principal Engineer"}
)
long_term_store.put(
    namespace=("users", "engineer_01", "episodic_memory"),
    key=str(uuid.uuid4()),
    value={"content": "Problem: ECS task OOM during batch run | Solution: Increased Fargate memory to 4GB and reduced chunk size to 500 records."}
)

Working memory is a four-line TypedDict. The add_messages reducer in the Annotated type is what makes it behave correctly: node returns are appended to the message list rather than replacing it, and messages sharing an ID are updated in place. Swap that reducer out and every node would clobber the entire conversation.

The seeded records are worth reading closely, because they show the two shapes long-term memory takes. The profile is a structured dictionary of typed fields, fetched by key. The episode is a single flat content string — deliberately, since content is the field being embedded, and cramming the problem and its solution into one string means a similarity hit on the problem retrieves the fix along with it.

The reasoning node: assembling context

# --- AGENT REASONING NODE ---
model = ChatOpenAI(model="gpt-4o", temperature=0).bind_tools(tools)

def agent_node(state: AgentState, config: dict, *, store: BaseStore) -> dict:
    user_id = config["configurable"].get("user_id", "default_user")
    latest_query = state["messages"][-1].content

    # Retrieve Semantic Memory (Profile)
    profile_item = store.get(("users", user_id, "semantic_profile"), "profile")
    user_profile = profile_item.value if profile_item else {}

    # Retrieve Episodic Memory (Semantic Vector Search)
    past_episodes = store.search(
        namespace=("users", user_id, "episodic_memory"),
        query=latest_query,
        limit=2
    )
    episodes_str = "\n".join([f"- {ep.value.get('content')}" for ep in past_episodes]) or "None found."

    # Construct context-enriched working prompt
    full_prompt = (
        f"{PROCEDURAL_GUIDELINES}\n\n"
        f"--- USER PROFILE (Semantic Memory) ---\n{user_profile}\n\n"
        f"--- RELEVANT EPISODES (Episodic Memory) ---\n{episodes_str}\n"
    )

    response = model.invoke([SystemMessage(content=full_prompt)] + state["messages"])
    return {"messages": [response]}

This function is the diagram’s centre box in code. Every arrow pointing into working memory shows up here as a line.

The two retrievals use deliberately different access patterns — store.get() for the profile because you know the key, store.search() for episodes because you do not. The search query is the user’s latest message, so recall is driven by whatever they just asked about.

limit=2 is a context-budget decision, not an arbitrary number. Episodes are unbounded and growing; the prompt is not. Two is a reasonable ceiling for keeping the most relevant history in-context without crowding out the actual conversation.

Two robustness details that are easy to miss: or "None found." guarantees the prompt section is never blank, which keeps the format stable on a cold-start user. And the final invoke prepends a freshly built SystemMessage to the existing message history rather than mutating state — memory context is reassembled on every single turn, so it always reflects the current store.

Wiring the graph

# --- GRAPH COMPILATION ---
builder = StateGraph(AgentState)
builder.add_node("agent", agent_node)
builder.add_node("tools", ToolNode(tools))

builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", tools_condition)
builder.add_edge("tools", "agent")

short_term_checkpointer = MemorySaver()
app = builder.compile(checkpointer=short_term_checkpointer, store=long_term_store)

The topology is the classic ReAct loop, and it is itself procedural memory — the graph structure encodes the permitted control flow. tools_condition inspects the model’s output and routes to the tool node if tool calls were emitted, or to END if not. The edge from tools back to agent closes the loop so the model can reason over its own observations.

The last line is where the checkpointer/store distinction becomes concrete: compile() takes both, as separate arguments, because they solve separate problems. checkpointer persists this thread. store persists everything that must outlive it.

Production hardening

The code above is a working reference, not a production deployment. Three things need attention before it becomes one.

Namespacing strategy. Maintain distinct namespaces for distinct data scopes — ("users", user_id, "profile") for personal data, ("teams", org_id, "policies") for shared knowledge. The namespace hierarchy is your access-control boundary, so design it before you have data in it. Retrofitting a namespace scheme onto a populated store is a migration.

Memory eviction and compaction. Episodic stores grow without bound, and an unbounded store gets slower and noisier simultaneously — more low-value episodes competing for the same top-k slots. Prune low-utility episodes, or aggregate repeated runs into generalised SOP documents. Compaction is not merely a cost measure; consolidating fifty near-identical incidents into one procedure genuinely improves retrieval quality.

Asynchronous store operations. InMemoryStore is for development. Under real throughput, use AsyncPostgresStore with connection pooling — every memory search is network I/O plus an embedding call, and doing that synchronously blocks the event loop on each turn.

Where the research is going: arXiv, mid-2026

The architecture above is the stable, buildable core. The research frontier has moved past it — and interestingly, almost none of the recent work is about better embeddings. Nearly all of it is about the parts this post flagged as “production hardening”: what you admit into memory, what you do when stored facts go stale, and what happens when someone writes to your store on purpose.

Here is what has landed on arXiv over the past few months. These are preprints, so treat the numbers as directional rather than settled.

1. Consolidation is moving to write time

The reference implementation above writes whatever the agent decides to write. The emerging consensus is that this is the wrong place to make the decision — filtering at read time means you have already paid the storage and retrieval-noise cost.

Dual-Layer Agentic Memory with Fast Write Routing and Slow Consolidation (arXiv:2608.22215) argues the core challenge is not retrieval but managing the knowledge lifecycle: deciding what to externalise, update, or eventually internalise. It classifies incoming information as non-write, write-new, or write-update, routed through a small-to-large model cascade so the triage itself stays cheap, with periodic parametric consolidation behind it. The framing is borrowed from Complementary Learning Systems theory in neuroscience — fast episodic capture, slow consolidation into generalised knowledge.

MemGuard (arXiv:2608.21867) attacks the same problem from the admission side, naming two failure modes: unreliable admission (failed trajectories and accidental successes enter memory because they look relevant, then mislead later decisions) and memory drift (banks accumulate duplicate, stale, and conflicting records that retrieval alone cannot repair). Its move is to keep verifier output as persistent lifecycle metadata — reward, confidence, label, uncertainty attached to every record — rather than using it as a one-shot filter at the door.

HiPS (arXiv:2608.25329) makes the retain/compress/discard policy itself learned rather than fixed, splitting it into a globally shared foundation and a user-specific adaptive tier — on the observation that the optimal memory decision is user-specific and shifts as the policy trains.

2. Staleness, not recall, is the dominant failure mode

This is the most consistent theme in the recent literature, and it maps directly onto the read-modify-write pattern in the update_user_profile_fact tool above. Overwriting a fact is easy. Knowing that a stored fact has been superseded is not.

Can Agent Memory Systems Track Evolving State? (arXiv:2608.19652) makes the argument explicitly: existing memory benchmarks are recall-shaped, but a useful memory system has to track the evolving state of the world, so that answers reflect current facts rather than superseded ones. The paper introduces StateMemBench, 234 multi-session scenarios whose grading distinguishes an answer reflecting the current state from one reflecting a superseded state — separating state-tracking failures from ordinary retrieval errors by construction. It reports the task as hard for existing memory systems, retrieval-augmented baselines, and long-context baselines alike.

When Stale Constraints Go Unchecked (arXiv:2608.25553) studies the inherited-memory version: a constraint that was true when written and has since been withdrawn by a newer authoritative record. Under a fixed two-record verification budget, agents inspected the constraint’s provenance path in roughly one episode in five, and when the constraint had been superseded, native budget allocation produced stale-consistent decisions around 75% of the time. The instructive part is that the failure was largely fixable within the same budget by re-assigning one slot to the provenance path — this is an allocation problem, not a capacity problem.

Remember, Verify, or Ask? (arXiv:2608.19564) formalises the decision boundary the consolidation tools above quietly skip: should interaction-derived information be persisted, used only in the current context, re-verified, or clarified with the user? Across model families it finds a consistent asymmetry — models verify changing facts far more reliably than they ask the user to resolve genuine ambiguity.

3. Persistent memory is an attack surface

Once memory is durable and retrieved automatically, a single poisoned write becomes a persistent behavioural change. This is the security consequence of everything else in this post, and the results are not comfortable.

Utility Under Attack (arXiv:2608.21230) uses plainly worded false assertions — no prompt-injection tricks, no trigger words, no retriever optimisation. Poisoning 1.2% of a LongMemEval corpus dropped accuracy from 0.850 to 0.300. More pointedly: a four-stage write-time screening pipeline that achieved 0.832 recall against indirect prompt injection rejected zero of 360 poisoned memories. The conclusion is a real boundary on content-only screening — telling a false assertion from a true one generally requires grounding outside the text itself. Provenance-weighted retrieval at its shipped weight was statistically indistinguishable from no defence at all.

InjecMEM (arXiv:2608.23471) shows the offensive side needs very little: a single ordinary interaction, with no read or edit access to the memory store, is enough to plant a record that steers later responses on a chosen topic. The construction pairs a retriever-agnostic “anchor” carrying high-recall topical cues, so retrieval reliably surfaces the record, with a short adversarial command that survives being fused into unpredictable context positions.

The practical reading for anyone building on the architecture above: an agent’s own consolidation tools are a write path from untrusted conversation directly into durable state. Namespacing gives you isolation between users; it gives you nothing against a user poisoning their own memory, or against content the agent ingested from a tool call.

4. Compaction destroys exactly the wrong things

The Compaction Cliff in Long-Running AI Agent Memory (arXiv:2608.22752) is the sharpest result here, and it concerns the procedural tier. A safety rule and an episodic log compete for the same context tokens — but when the budget overflows, both get summarised at the same rate, and only the rule needs exact wording to stay enforceable. Measuring across 20 production agent configurations, the paper reports Claude Code’s /compact on Sonnet 4.6 preserving 53% of safety rules after a single compaction round, and 10% after five.

Their fix, Knowledge Triage, is the same instinct as the four-tier split this post argues for, applied one level down: classify each line of the knowledge base by type and give each type its own retention policy, with operators that rewrite in place under per-type fidelity, partition topics too large to compact safely (replicating in-scope safety rules across partitions), and page items back in from external storage.

5. Retrieval is growing structure and state

Flat vector search over a growing pile is being replaced by stores that know something about their own contents.

• Weighted Memory Tree (arXiv:2608.20631) organises execution into tasks, subtasks, and actions, and gives every memory a dynamic retention score updated by events and selection-based decay — so completed trajectories fold away and low-utility content is suppressed without being lost.
• EARM (arXiv:2608.22767) points out that agents accumulate memories but their retrievers never accumulate retrieval experience. It stores past LLM relevance scores in a sparse online matrix and completes it causally, so expensive reranker judgements are amortised across queries instead of discarded after each one.
• HERO (arXiv:2608.22310) targets the two costs of the compress-and-rewrite approach — information loss from compression and semantic drift from rewriting — by converting dialogue history into a traceable heterogeneous memory graph that keeps the raw text as evidence.
• PolyMemDB (arXiv:2608.25577) rejects the single-store paradigm outright, running graph, vector, probabilistic, and spatio-temporal storage side by side with a probabilistic inference engine that uses temporal decay and provenance to resolve long-term factual conflicts.

6. The benchmarks are getting harder — and less flattering

DreamBench-SWE (arXiv:2608.20664) tests multi-session memory hygiene for software agents, where later tasks depend on non-inferable evidence from earlier sessions and are graded by executable hidden oracles. Its headline comparison is a useful splash of cold water: no external memory passed 21/180 units, while deterministic verbatim event memory reached 82/180. Sophisticated memory systems did not clearly beat simply keeping an accurate log.

That result deserves to sit next to the rest. Before reaching for graph stores and learned retention policies, it is worth checking whether faithfully recording what happened — and not summarising it into vagueness — gets you most of the way there.

What this changes about the reference architecture

Nothing in the implementation above is invalidated by this work; the four-tier split holds up, and the checkpointer/store distinction is if anything more important. But three additions look increasingly non-optional for production:

• Provenance on every record — a timestamp, a source, and a supersession pointer, so “is this still true?” is answerable without re-deriving it. Nearly every staleness paper here converges on this.
• A write policy, not just a write tool — triage at admission time rather than trusting whatever the agent decides to persist.
• Type-aware compaction — procedural rules must not be summarised on the same schedule, or at the same fidelity, as episodic logs.

Current research areas

A note on method, because it turned out to matter. The section above came from a recency-ranked sweep of the arXiv API. Running the same subject through alphaXiv’s community-ranked search returned a substantially different set — and the papers it surfaced that the first sweep missed are, on the whole, the more important ones. Sorting by “newest” buries a survey from two months ago under a hundred preprints from last week. Both rankings are worth running.

Start here: the systematic survey

Are We Ready For An Agent-Native Memory System? (arXiv:2606.24775) is the single most useful thing to read on this topic, and it is the paper that most directly stress-tests the architecture in this post. It evaluates 12 representative memory systems plus two baselines across five workloads spanning 11 datasets, and — critically — it decomposes each system into four modules rather than treating it as a black box: representation and storage, extraction, retrieval and routing, and maintenance.

It also draws a clean boundary that is often blurred: agent memory is not RAG and not context engineering. RAG is stateless and read-only over a static corpus; context engineering curates the window for one turn; agent memory is a persistent, updatable substrate managing agent-specific state over time. It further argues these workloads are unlike traditional OLTP/OLAP — access is semantic rather than predicate-based, and contents evolve under conflicting observations.

Of its nine findings, four bear directly on decisions in this post:

• No architecture dominates. Effectiveness depends on whether the memory structure matches the workload’s actual bottleneck. Relation- and time-aware retrieval wins on dispersed cross-session reasoning; summary-first routing wins on long coherent dialogue; preserving raw interaction traces wins for stateful execution.
• Preserving evidence beats abstraction. High-retention forms best support exact recall; hierarchy improves access but “cannot restore content removed during representation.” Compression is a one-way door.
• Filter late, not early. Extraction should preserve context at write time; aggressive filtering discards cues that turn out to matter later.
• Cost is governed by the scope of maintenance operations, not by whether you have structure. Localized updates give the best cost-utility balance; rich organization pays off only if upkeep avoids broad recomputation.

Which substrate, under which regime

Harness the Memory (arXiv:2608.15008) holds everything constant except the memory substrate — 11 methods across seven substrate families, three backbones, four benchmarks, 26 instrumented metrics — and finds the ranking reverses between regimes. Structural graphs and hierarchical memories lead on user-centric QA; refinement memories that distill trajectories into reusable strategies lead on agentic planning, where graphs are Pareto-dominated.

Its sharpest result concerns retrieval breadth. Increasing k helps factual QA and hurts agentic tasks — attention probing shows why: as k grows, attention mass shifts from the current observation and action list toward the retrieved block. For QA the answer lives in that block, so the shift helps. For an agent mid-task it is dilution, and success drops while steps-to-goal rises. Their design rule is worth adopting verbatim: trade read breadth for write depth.

Parametric memory: the axis this post left out

Everything in the reference implementation is non-parametric — memory lives in a store, not in weights. Three papers argue that is only half the design space.

Memory Decoder at Scale (arXiv:2607.27919) pretrains a dedicated memory module that emulates a kNN retriever’s output distribution, scaled to 6.9B parameters over 300B tokens. The headline is a parameter-efficiency result: a Pythia-410M backbone with a 6.9B memory scored 37.34 against a full Pythia-12B’s 37.24 — matching it with 39% fewer total parameters. Domain memories of 1.7B beat continued pretraining, LoRA, and RAG by at least 4 points, and transfer across model families with different vocabularies at 20% of the usual training budget. Memory and reasoning, it turns out, do not have to share a parameter set.

UniMem (arXiv:2607.26017) bridges the two worlds: novel queries go to an episodic buffer served by retrieval, and only once a recurring pattern accumulates enough evidence is it consolidated into a dedicated parametric block. Over 16,000 streaming samples covering 100 novel tasks it created 76 parametric units — merging similar tasks and leaving 10 sparse ones in the buffer rather than allocating parameters prematurely. This is the episodic→semantic consolidation arrow from the diagram, implemented as an actual weight update.

Consolidator (arXiv:2608.11701) is a mechanism-level proof of concept with one finding worth carrying forward conceptually. Training only 12.35K parameters — 0.041% of the model, everything else frozen — it tests whether consolidated long-term memory is merely retrievable content or also an access state that shapes which memory slots later inputs reach. Feeding retained memory into the router lifted updated-mapping recall from 44.4% to 87.0%, while immediate short-term recall stayed identical at 89.9%. Memory that only answers queries is doing half the job; memory that also steers retrieval is doing the other half.

Consolidation granularity and the cost of forgetting

LycheeMemory V2 (arXiv:2608.12990) attacks the write-side cost of eager consolidation — invoking an LLM after every turn. It batches exchanges into semantically coherent segments using embedding-based boundary detection, then consolidates once per segment. On LoCoMo that cut construction tokens by 86% versus A-Mem while raising accuracy to 89.22%, and query tokens fell too. The ablation is the interesting part: reverting to turn-level consolidation cost 7.3 points of accuracy and raised construction tokens 316%, and replacing semantic boundaries with fixed windows cost 6.8 points. Batching buys the cost saving; semantic boundaries buy the accuracy.

What to Keep, What to Forget (arXiv:2607.08032) unifies KV-cache compression, prompt compression, architectural state compression, and agent memory consolidation as a single rate–distortion problem, with a Fano-style lower bound saying that below a task’s inherent information requirement, errors are unavoidable at any layer. Its empirical result is the one to remember: comparing a reversible operator (archive and retrieve) against an irreversible one (LLM summarization) over repeated compaction events, the reversible operator held recall near 0.95 while the irreversible one sat at 0.33–0.56 and degraded further with each round. Their first design principle follows directly — never irreversibly discard what cannot be cheaply re-derived.

Retrieval as recollection

RippleMem (arXiv:2608.13334) reframes the access problem: the bottleneck is not storage but recovering a full evidence set when the relevant pieces are scattered across many interactions. Flat retrieval returns isolated, incomplete records. RippleMem recalls anchors through hybrid cues, then expands outward along semantic and structural associations — initially recalled memories act as cues for completing the rest. It reports +3.95% on LoCoMo and up to +11.87% on LongMemEval-S while cutting graph construction cost roughly 30×.

Caching for the Future (arXiv:2608.04746) takes its principle from western scrub jays, which recover perishable worms when caches are fresh and durable peanuts when they are not. The observation driving it is precise: a user’s profession stays valid for years, their meeting room is stale by afternoon, their current branch may not survive lunch — and uniform age-based discounting fails because memory types decay at fundamentally different rates. ScrubJay-MEM binds each memory as a What–Where–When tuple with an estimated perishability coefficient and utility horizon. Notably, the authors scope their own claim: gains narrow under stronger backbones and reverse on fact-consolidation tasks.

Security, revisited: repair and authority

The section above covered attacks and failed defences. Two papers cover what comes after.

MemSecBench (arXiv:2607.27080) traces the same malicious content through a full Write–Execute–Forget lifecycle across 310 cases and 24 agent configurations. Malicious content persisted in 84.2% of cases, the full write-then-exploit chain succeeded in 50.3%, and selective repair succeeded in only 56.1%. Two bottlenecks emerge. On the attack side it is adoption — recall stayed high at 76.1% but adoption dropped to 53.7%, making the moment an agent acts on recalled memory the best place to intervene. On the repair side the bottleneck is not removing the bad record (86.3% succeeded) but doing so without destroying good ones (62.5%). No backend was uniformly safer, and resistance and recoverability turned out to be independent properties: one swap cut attack success barely at all while lifting repair from 46.3% to 87.6%.

When Memory Becomes Authority (arXiv:2608.01679) identifies a failure this post’s update_user_profile_fact tool is squarely vulnerable to. Consolidation preserves the content of a claim while erasing the source constraints governing its use — “the user lives in Seattle” survives, but whether the user said it or a third party merely reported it does not. The authors call the resulting false upgrade authority laundering, and found it in 48 of 49 consolidator-backend configurations.

The consequences are measurable. With source-erased memory, agents performed prohibited actions 50.3% of the time and permitted ones 49.4% — statistically indistinguishable, meaning authority was simply not a factor in the decision. Natural-language provenance in the memory text helped only modestly (down to 40.5%). Structured authority metadata cut it to 5.8%, and an end-to-end pipeline with automatically predicted labels took unauthorized actions from 16.9% to 0.0% with no loss of legitimate task success. The lesson is specific: provenance has to be a typed field the agent can condition on, not a sentence in the memory body.

The short version

If you read three of these, make it the agent-native memory survey for the empirical grounding, the rate–distortion paper for the reversibility principle, and the authority-collapse paper for the security model. Taken together with the section above, they point at a consistent conclusion: the hard parts of agent memory are not retrieval quality. They are deciding what to write, keeping what you wrote honest about where it came from and whether it is still true, and never destroying anything you cannot rebuild.

Closing thought

The temptation with agent memory is to treat it as one problem and solve it with one tool — usually a vector database that everything gets dumped into. The four-tier split exists because these are genuinely four different problems with four different access patterns: state you checkpoint, experience you search by similarity, facts you fetch by key, and rules you version-control.

Get the split right and the retrieval logic becomes almost boring — a get here, a search there, assembled into a prompt. Get it wrong and no amount of embedding tuning will save you, because you will be running similarity search over things you should simply have looked up.