AI agent memory, what to persist

“Add memory” can describe several different requirements. What people mean varies wildly: keep the thread, remember my preferences, stop asking me the same question, know what happened last week.

Those are four different systems with four different lifetimes. Building them as one vector store is why agent memory so often feels both forgetful and stubbornly wrong at the same time.

Four tiers, four lifetimes

1. Scratchpad. The current reasoning trail. Lives for one run, then dies. Never persisted beyond logs.
2. Task state. The objective, progress, blockers, decisions, and identifiers for the job in flight. Lives until the task closes. Belongs in a real table with a real status field.
3. Durable facts. Preferences, entitlements, relationships, stable attributes. Lives indefinitely, updated in place, always attributable to a source.
4. History. What happened, when, and by whose instruction. Append-only, long retention, read for audit rather than for reasoning.

Task state is the tier people skip

Start by checking whether task state is explicit. An agent can lose task state when it carries that state only implicitly, in the conversation, where it competes with everything else for attention and gets compressed away.

Make it explicit. A small structured object, updated as the run proceeds and injected near the top of context on every turn:

• Objective, in one sentence.
• Steps completed, with outcomes.
• Constraints discovered along the way.
• Approaches already tried and failed.
• Open questions and current blocker.
• Identifiers touched, so nothing gets acted on twice.

That last two lines eliminate the most irritating agent behavior there is, which is retrying something that already failed and re-processing a record it already handled. The context mechanics of injecting this cleanly are in context engineering is the whole job now.

Durable facts belong in tables, not embeddings

This is where the vector store gets misused. Semantic search is excellent at "have we discussed something like this before" and unreliable at "what is this customer's billing plan". The second question needs an exact answer with a known provenance.

• Typed fields for anything that drives a decision. Plan, tier, locale, timezone, contact preference, entitlements.
• A source and timestamp per fact. Stated by the user, inferred by the agent, or synced from a system of record. Treat these differently.
• Update in place, never append. Two contradictory stored facts can lead to inconsistent answers.
• User-visible and editable. If a person cannot see what the system believes about them and correct it, you will be debugging by archaeology.

Write memory deliberately, with a tool

Automatic inference into long-term storage sounds delightful and creates confident nonsense. One offhand comment becomes a permanent belief that shapes every future interaction.

Make remembering an explicit tool call with a narrow schema. The agent must state the field, the value, and the source. Validate it. Log it. Then a wrong memory is one row you can find and fix, instead of a vibe buried in an embedding.

It was right once, four months earlier. Nothing in the design allowed that belief to be updated.

Retrieval over history, done carefully

Semantic recall over past interactions is genuinely useful, with three guards:

1. Recency weighting. A relevant conversation from yesterday outranks a slightly more similar one from a year ago.
2. Summaries, not transcripts. Index a structured summary of each interaction. Raw transcripts retrieve noisily and cost more.
3. Hard scoping. Filter by user and tenant in the query, enforced by the database. Cross-tenant memory leakage is the worst bug in this category and it is entirely preventable.

Eviction is a feature

Every tier needs a written rule for what leaves and when. Without one, context grows, cost grows, quality drops, and your privacy posture quietly deteriorates.

• Scratchpad: discarded at end of run, sampled into logs only.
• Task state: archived on task close, retained as history.
• Durable facts: reviewed on staleness, expired where they can go stale.
• History: full payloads on short retention, structured summaries and aggregates on long retention, per observability practice.

Also give users a delete path that actually deletes, across all four tiers including the vector index. If your answer to a deletion request involves the word "eventually", you have a compliance problem rather than a memory system.

A starting memory design

A tasks table with a JSON state column. A facts table with typed columns, source, and updated timestamp. An append-only events table for history. A vector index over interaction summaries, scoped by tenant and user. One explicit remember tool with a schema.

The schema, retention rules and update paths need implementation and testing. Include their storage and maintenance costs in the design.

If an agent forgets decisions or repeats work, inspect its stored state and update rules. If you have a stalled product build that has to work for a real customer, pilot, or launch, send the project brief. Scope and price are agreed before work starts.

Frequently asked