Why does AI forget what you told it?

The short version

Why does AI sometimes remember something from long ago, then forget something you told it a moment ago? It isn't random. Something is happening behind the scenes that decides what it knows about you and what it doesn't, and you can control a good part of it.

In this lesson we tell three things apart: Context, everything the model can see in front of it right now; the Context Window, the most it can see at once; and Memory, the place where the app keeps information about you to use in a new chat. Then we see the difference between them in practice, in Claude.

Key terms

• Context: all the information the Model can see in front of it right now, and relies on to reply to you.
• Context Window: the largest amount of text the Model can see at once, measured in Tokens, not words.
• Memory: a place where the app keeps information about you, so it can use it in a new chat.

The explanation

The clerk's desk

We continue with the same picture from the earlier lessons: the Model is a clerk sitting behind a counter at a government office. Every time you write them something, the slip is placed on the desk in front of them. Their own replies become part of the papers in front of them too.

This clerk doesn't remember anything on their own. Before every reply, they read all the papers on the desk from the top, as if seeing them for the first time.

But the desk has limited space. When it fills up, an old paper has to come off for a new one to go on.

And if you open a new chat? Same clerk, but the desk is empty. The papers you handed in before are no longer in front of them.

The small safe beside the desk

Sometimes the clerk has a small safe beside the desk, holding a few notes about you from earlier visits. If one of them is useful, they take it out of the safe and put it on the desk before you hand in anything.

Now let's call things by their names:

• The papers on the desk are the Context: all the information the Model can see in front of it right now, and relies on to reply to you.
• The small safe is Memory: a place where the app keeps information about you, to use in a new chat.
• The space on the desk is the Context Window: the largest amount of text the Model can see at once.

That amount isn't measured in words but in Tokens: small pieces of text the model reads and writes in, as we saw in an earlier lesson.

When the desk fills up

The longer the chat, the bigger the Context grows, until it fills the Context Window. From there, every app handles it its own way:

• Some apps drop the start of the conversation from the Context.
• Some ask you to start a new chat.
• Some compress the Context (Compact), meaning they summarize it.

In every case: what isn't in the Context, the Model can't see.

But note: a new chat doesn't always mean the Context is completely empty, because Memory may add some information about you to it before you type your first message.

The experiment in Claude

First we took Memory out of the experiment, to see the effect of Context alone. Under Settings, then Memory, we chose Pause memory: Claude keeps any information saved before, but doesn't use it in the experiment, and doesn't add anything new to Memory. Paid accounts also have a Search and reference chats option, which we turned off so Claude wouldn't search old chats.

Then we opened a new chat and told Claude something about ourselves: that we have a cat called Mishmish. Then we asked it the cat's name. It answered correctly, which is expected, because the information was still in front of it in the same chat, in the Context.

Then we opened another new chat, on the same account, and asked the exact same question. This time it didn't know the answer, and pointed out that memory was turned off. The information isn't in the new chat's Context, and when it tried to look in Memory, it found it paused.

If you want Claude to know it in a new chat, the information has to reach it again in one of two ways:

• Write it in the new chat, so it goes straight into the Context.
• Save it in Memory, so Claude uses it from then on when it's relevant.

So we tried the second way: we turned Memory back on, and left Search and reference chats off to be sure the answer came from Memory alone. We wrote the information into the memory field and waited until it was saved. To be extra sure, we deleted the old chats where we had mentioned the cat's name. Then we opened a new chat and asked the same question, and this time it knew: "Your cat's name is Mishmish."

Why? Because the information was saved in Memory, so Claude added it to the Context while forming its reply.

Not just in Claude

The idea of Memory exists in many tools, such as ChatGPT and Perplexity, but how it works and what it looks like differ a little from tool to tool.

Try it yourself

Suggest something that would make my work easier

Why this matters to you

You'll meet the same idea when we build things with tools like Claude Code: the tool can't see what isn't in the Context. So one of the skills we'll learn is putting exactly what the Model needs in front of it. Give it less, and it will guess. Give it more, and the important part may get lost among the rest.

Why not give it everything and let it pick what it needs? Because the more the Context fills with things that have nothing to do with the task, the harder it is for it to focus on what matters. A well-chosen Context beats a big one.

Related lessons

• What is AI?: what Model means, and the government office counter.
• How does AI think?: what a Token is, the unit the Context Window is measured in.
• Prompt: how do you ask for exactly what you want?: how to write what the model needs into your request.