Skip to content
Interface language: English
Audit / Signal

AI assistant memory: how to check what was remembered and why

An assistant may store different information from its chat history. Test sources, access, updates and deletion in a controlled trial.

Article language: English

Illustration: part of a stone column, with a thin line marking a crack.
In this article

An assistant remembers that a project ends on Friday. Friday has passed and the deadline has moved. Memory makes work easier to continue, but it can carry an old agreement into a new decision. An audit should establish what is stored, where it came from, where it may be used and how information can be corrected or removed effectively.

Start by distinguishing memory, conversation history and current context: the material available for a particular answer. Feature names differ across products, so describe the actual mechanism and observed behaviour. Asking “what do you remember about me?” does not prove the list is complete or identify every storage location.

Establish what the product calls memory

It may be a separate preference record, selected passages from past conversations, a file maintained by an agent or information fetched from a connected service. These mechanisms may operate together but have different sources, permissions and deletion rules. Describe them separately in the register.

ChatGPT memory documentation notes that features and settings depend on plan, region, platform and workspace configuration. It also says a memory summary need not include every item that can affect personalisation. Inspect the right account and its visible settings.

In your AI tool inventory, record who uses memory and for which tasks. The guide to taking an inventory of AI tools (in Polish) explains how to organise the record. A person's private preferences and a team process's memory should not share one description merely because the same product supports both.

Trace storage locations and access

Identify where a record resides, who reads it and how it reaches an answer. A company-built system may use its own database or file directory. In a finished product, part of the flow may only be visible through provider settings and documentation. Record that limit of the assessment.

For example, the Claude memory tool operates on the application side: the model requests file operations and the application performs them and controls storage. This describes a specific API tool, not proof that every Claude-based product stores memory alike.

Connect these findings to your AI data-flow register. The guide to recording AI data flows (in Polish) explains how to trace the steps. Besides memory, the original conversation, a file copy and an output exported to another system may exist. Removing one record does not describe what happens to the others.

Test with an easily recognised fact

In a controlled task use an artificial project name and an agreed sample date. Request storage through the available feature, then check a new conversation or session. Record whether the information returned, how it was used and whether its source can be identified.

Change one element, such as the date, and repeat. The model should distinguish the current agreement from the old record. If both remain available, examine how the conflict is resolved. A concise note without a date can act like a very confident but outdated calendar.

Record the starting state in the automation test environment, and restore it when repeated trials require the same conditions. Otherwise earlier records may make successive runs differ and the team may wrongly attribute the difference to the model itself.

For more on setting a recognisable starting point, see the AI automation test-environment guide (in Polish).

Memory should support the source, not replace it

“Client prefers the basic option” can be a useful shorthand. It does not confirm a current purchase decision or agreed terms. For an important action, the process should return to the proper source and check its currency.

When generating sales proposals with AI, check whether an old discount or date enters a new proposal as a default agreement. Memory can suggest a question for the account owner, but should not independently renew company commitments. The AI sales proposal guide (in Polish) distinguishes a customer request from a confirmed commitment.

If the assistant uses company search, separately assess the sources behind its RAG answers and whether those answers are current. The article on RAG sources and current answers (in Polish) distinguishes finding a source from drawing a conclusion. A relevant search result can compete with an old memory shorthand. Check whether the answer preserves the appropriate hierarchy of evidence.

Examine boundaries between people and projects

Use two artificial projects with similar names. Save a different preference or date in one. Check whether the other receives it and whether access matches the designed scope. Do not use another team's real confidential material for this.

The review of AI integration permissions also covers memory managed by the application. Separate folders are not enough if the operation's executor can reach every folder without checking identity and task scope.

The article on AI integration permissions (in Polish) discusses identities, accessible resources and access rights.

In your company AI policy, state what kinds of information may be retained and who may change shared process memory. The guide to a company AI-use policy (in Polish) explains how to develop such rules. Not every person's preference should become a team-wide rule. The user should see whether a record concerns a project, person or shared rule and know where to report an inappropriate extension.

Check deletion separately for each mechanism

ChatGPT documentation explains that deleting a conversation need not remove separately saved memory, and turning memory off does not remove conversation history. Details depend on available personalisation and sources. Test the relevant controls rather than treating one action as a global wipe.

A custom system needs deletion rules and confirmation at the storage location. Then run a new task and check use of the control fact. Its absence from one answer does not prove every copy was deleted. Combine behavioural observation with available technical evidence.

The article on AI data retention and deletion (in Polish) helps distinguish memory, source and other copies. If a product cannot be examined to the required extent, name the limitation and refer it to the use-case owner. Do not promise complete deletion based on an empty screen.

Aeneia conceptual photography

Acceptance should define memory's permitted role

Record permitted purposes: recalling preferences, finding materials or continuing a working task. For each, set decision boundaries and when to check again. A model change, integration change or team change may require another trial.

If unauthorised data use is uncovered, follow the agreed AI data incident response procedure. Do not continue experimenting with real material merely to obtain a clearer example. Further action should clarify and contain the incident.

The guide to an AI data incident (in Polish) discusses establishing facts and limiting the impact.

Distinguish a record from an inference about a person or project

An assistant may receive information directly or generalise from several statements. “For this proposal we choose a shorter description” does not establish a permanent preference for every document. Check whether storage preserves the situational scope or turns a one-off request into a rule. An overbroad note may later influence tasks the author never intended.

In a controlled trial provide a fact, a conditional proposal and a rejected idea. Then inspect what was saved and how it returns in a new task. The record should distinguish the three. Calling something an “agreement” does not establish that the authorised person decided it. Memory should not promote an idea into an approved task.

For important notes retain source, date and owner. A review date can be useful, particularly for deadlines and case status. Not every preference needs a formal record, but information driving action should have a verifiable basis. Otherwise the team cannot resolve a conflict between a current document and an old shorthand.

Check who approves changes to shared memory. One person may prepare a proposal while the process owner adopts a team-wide rule. If storage is automatic, define a limited scope and a correction mechanism. Do not rely only on an assumption that the model always recognises which statement deserves permanent retention.

Also distinguish stored content from permission. Remembering that an owner once sent a file to a person does not authorise another send. When acting, the application must check the current recipient, resource and authorisation. History helps explain the case but does not itself expand the permitted operation.

Check whether a correction reaches each place of use

Changing a note may not remove information from the current conversation, an earlier summary or a copy given to another agent. When several agents collaborate, examine handover of conditions and evidence between agents to establish which task version the next role uses. For a way to locate a condition lost between roles, see the agent handover article (in Polish). Trace these places in the actual workflow. Do not assume a visible field update refreshes every context. The user needs to know which tasks must be prepared again after an important agreement changes. If the process reuses a completed output, also examine the AI answer cache. This separate mechanism may return an old answer after the note has been corrected.

The AI caching article (in Polish) also distinguishes saved answers from reuse of a processed prompt prefix.

In a trial save an artificial deadline, start a task and then correct it. Check both the work already under way and a new session. Decide whether the process should stop the running task or finish it while identifying the older version. The answer depends on the use case, but should not be an accidental consequence of when the model fetched material.

For memory shared by several people, check concurrent conflicting changes. Two similarly titled notes need not concern the same decision. Define case identification and history preservation. Automatically retaining only the last write can remove a needed limitation or attach a date to the wrong project.

After a correction retain a brief record of its scope: who changed the information, why and where the new state was confirmed. This does not require copying an entire confidential conversation into a report. Appropriate identifiers and source access for authorised people suffice. The record should help explain discrepancies, not create another storage location for content.

Plan for memory access failure. An assistant should recognise that it did not receive a record instead of presenting guessed preferences as known agreements. The process can request confirmation or work within a reduced scope. An empty archive shelf does not entitle anyone to write their own version of project history.

Decide whether the task needs memory

Not every task needs persistent storage. A one-off comparison of artificial options may use current material alone. Memory adds value when it reduces repeated agreements and helps find the right context. Test this benefit in a specific workflow rather than assuming that more retained information always improves work.

Prepare two variants of the same task: one with permitted memory and one with complete, deliberately prepared input without it. Compare correctness, preparation effort and corrections needed. This is not a feature contest. The trial should show whether retention is justified and which information actually helps continue work.

Also test a change of case owner. A successor should find the current source and understand the limits of stored shorthand. If only the previous person's private assistant remembers, the company may struggle to hand over the process. Decide which approved materials must be maintained independently of personal conversation history.

After the use ends, perform agreed actions on memory and connections. Retain evidence of the scope that can be confirmed and state provider limitations. Acceptance should distinguish actual deletion, loss of further access and non-use during the trial. These are three observations that may complement each other but are not the same evidence.

For process handover prepare a short list: which information is merely a hint, which points to an approved source and who resolves conflicts. An assistant user should understand the difference without knowing the technical data store. Memory is meant to help work, so its limits must be clear in an ordinary task.

Run a trial without the configuration author's hints. Ask someone to correct an artificial agreement and check it in a new conversation. Observe whether the participant knows where to make the change, what remains in history and when an administrator is needed. If they must repeatedly ask the model to truly forget, improve instructions for the actual action instead of relying on the assistant's declaration.

Do not judge memory by the number of notes collected. More useful is the ability to find the current basis and repair an incorrect shorthand. An overgrown record without an owner can act like a drawer full of notes: something will be found, but not always what currently applies. Acceptance should show how the team maintains order.

To assess memory in a specific workflow, see Aeneia Audit.

Aeneia / Aeneia editorial

Editorial article. Source links are included in the text.

About Aeneia

Brands we have worked with

  • L’Oréal
  • Garnier
  • Netflix
  • Carrefour
  • Warner Bros. Discovery
  • Biedronka
  • HBO Max
  • Wolt
  • Żabka
  • COSRX
  • Embassy of China in Poland
  • Mixa
Aeneia / Initium

From knowledge to a decision

Tell us what you want to put in order. We will agree on where to start.

Contact Aeneia

We partner with and use solutions from:

  • Google
  • OpenAI
  • Microsoft
  • Stripe
  • Anthropic
  • Linear