Applied cognitive architecture
Your AI pilot did not fail because the model was bad. It failed because you only gave it one of three memories.
The pattern is consistent enough to have a name in the research literature. A company connects a model to its documents, gets answers that are technically correct and operationally useless, and concludes that AI is not ready. The documents were never the problem.
Written by Kristian Baer, Northtek · Anchorage, Alaska · Updated 2026-08-21
The short answer
Why do most business AI pilots fail?
Because a document store is only one of the three long-term memories an agent needs, and it is the least useful one on its own. The cognitive architecture literature splits an agent’s long-term memory into semantic memory (what is true - your specs, policies, and rate tables), episodic memory (what happened - your past jobs, tickets, and incidents), and procedural memory (how your organization actually does things - escalation paths, approval chains, who signs what). A typical pilot loads documents into a retrieval system, which supplies semantic memory and nothing else. The result is a system that can quote your policy but cannot tell you what you did the last time this situation came up, and cannot follow the sequence your team would have followed. The answers are correct and useless at the same time, which is the exact signature of missing episodic and procedural memory.
The framework
Language agents have a memory taxonomy, and it is borrowed from cognitive science.
In 2023 a group at Princeton published Cognitive Architectures for Language Agents, usually shortened to CoALA. Its most-cited contribution is a decomposition: a language agent has short-term working memory that tracks the current state, and long-term memory divided into semantic, episodic, and procedural stores. That three-way split is not new. It is Endel Tulving’s memory trichotomy from experimental psychology, imported into agent design because the same distinctions turn out to matter.
CoALA also splits what an agent can do into external actions, which touch the world, and internal actions, which touch its own memory: reasoning, retrieval, and learning. Hold on to that last one. Most deployed business tools implement retrieval and nothing else, which is why they never get better at your business no matter how long they run.
The mapping
Every business already has all three memories. Only one of them is in a document.
This is the part that does not appear in the papers, because the papers are about agents rather than about dispatch offices. Translate the taxonomy into an operating business and the diagnosis becomes obvious.
- Semantic memory - what is true
- Your specifications, contracts, rate tables, wage determinations, permits, policies. Already written down, already in files. This is the easiest memory to give an agent, which is exactly why every pilot starts here and why so many of them stop here.
- Episodic memory - what happened
- The job you did at that building in 2023. The claim that got paid short and why. The shareholder who called in March. This lives in your ticket history, your project records, your email, and your accounting system. It is not in a document you can upload, and retrieving it requires integration rather than ingestion.
- Procedural memory - how you actually do it
- Which exceptions go to the owner and which get handled. What has to be true before a change order goes out. Who gets called at two in the morning. This one is usually written down nowhere. It lives with the dispatcher who has been there fourteen years, and the only way to capture it is to watch the work.
The diagnostic
The symptom tells you which memory is missing.
Once you know there are three, the failure modes stop looking like one general disappointment and start looking like three specific gaps, each with a different fix.
- It quotes the policy but misses the situation
- You asked what to do about a specific customer and got the general rule. Missing episodic memory. The agent has no access to what happened the last four times, so it cannot recognize a pattern it has already seen.
- It gives a correct answer that nobody can act on
- The information is right and the next step is wrong or absent. Missing procedural memory. Nothing has told it that in your shop this needs the owner’s sign-off before it moves.
- It was impressive in the demo and mediocre by month three
- It only implements retrieval, not learning. Every correction your team makes is discarded. The system is exactly as good on day ninety as it was on day one, which in a business that changes is the same as getting worse.
The common wrong fix
A bigger context window is not more memory.
The reflex when a pilot underperforms is to reach for a model with a larger context window and paste more in. In CoALA terms the context window is working memory: the scratchpad for the current decision cycle. Expanding it does not create long-term memory, because nothing persists once the session ends. It also gets expensive precisely on the high-repetition workflows that were worth automating in the first place, and recall across a very long context is not uniform, so the middle of what you pasted may effectively not be there at all.
The fix for missing long-term memory is architecture: a store that persists between sessions, holds episodes as well as facts, and gets written to when something is learned. That is a build, not a setting.
Why we work the way we do
This is why the first week is spent watching, not building.
Semantic memory can be collected by asking for a folder. Episodic memory can be collected by integrating with systems you already run. Procedural memory cannot be collected at all, because it does not exist in a recorded form - it has to be observed and then written down for the first time.
That is the entire reason a Northtek engagement starts with a week of sitting with the people who do the work and tracing real cases end to end, including the ones that went wrong. It is not a discovery ritual and it is not padding. It is the only extraction method that exists for the memory most pilots are missing.
Apply it
What to do with a pilot that is already underperforming
None of this requires starting over. It requires identifying which memory is absent and adding it, in this order.
-
01
Write down one real case end to end
Take a single recent situation your pilot handled badly. Write what a good employee would have done, step by step, including what they would have looked up and who they would have asked. That document is procedural memory that did not exist an hour ago.
-
02
Connect one system, not five
Pick the system holding the history most relevant to that case - the ticket log, the project record, the CRM. Episodic memory from one well-chosen source beats a shallow connection to everything.
-
03
Add a correction path
Give your team a way to mark an answer wrong, and make sure that mark changes future behavior. Without it you have retrieval, not learning, and the plateau is guaranteed.
-
04
Re-run the same case
Not a new demo. The same case that failed. If the answer is now actionable rather than merely accurate, the diagnosis was right and the rest is scale.
References
Read the primary sources
Everything asserted above traces to one of these. Where a claim is contested or the evidence is weaker than it is usually presented, we say so on the page rather than in a footnote.
-
01
Sumers, T., Yao, S., Narasimhan, K., Griffiths, T. (2023). Cognitive Architectures for Language Agents. Transactions on Machine Learning Research. arXiv:2309.02427.
The memory taxonomy and the internal action space used throughout this page.
-
02
Tulving, E. (1972). Episodic and Semantic Memory. In Organization of Memory, Academic Press.
The original distinction CoALA imports. Worth knowing that this idea is fifty years older than the current wave.
-
03
Northtek (2026). GENOME - an offline memory server for agents. Open source, benchmarked against Mem0 on public datasets.
Our implementation of persistent long-term memory. Harness and full results published, including where we only reach parity.
Straight answers
Is this just retrieval-augmented generation with extra steps?+
Retrieval-augmented generation is one mechanism, and it addresses semantic memory well. The point of the taxonomy is that semantic memory is the smallest part of the problem. Episodic memory requires reaching into operational systems rather than a document index, and procedural memory usually has to be written down for the first time. Calling all of it RAG hides the two gaps that actually cause the failures.
Our pilot uses a very capable model. Does the architecture really matter more?+
For this failure mode, yes. Model capability determines how well an answer is composed. Memory architecture determines whether the facts needed to compose a useful answer are available at all. A stronger model with no episodic memory produces a more articulate version of the same unusable answer.
How long does it take to add episodic memory?+
It depends entirely on whether the history is in a system with an interface. A ticketing platform or a CRM with an API is usually a week or two. History spread across email and a shared drive is a longer project, and we will tell you which one you have during the first week rather than after you have committed.
Can we capture procedural memory without bringing someone in?+
You can start. Have the person who handles exceptions narrate three recent ones into a voice memo, including the reasoning, then transcribe it. That is genuinely useful and it is free. What an outside observer adds is catching the steps the expert does not mention because they no longer notice doing them, which is most of the interesting ones.
Does this apply to a small business or only a large one?+
It applies wherever the work repeats. A four-person dispatch office has the same three memories as a corporation with a dozen subsidiaries, and the procedural memory in a small business is usually more concentrated - one person holds all of it, which makes capturing it both more valuable and more urgent.
Where does this framework come from?+
The taxonomy is from CoALA, published by a group at Princeton in 2023 and now a standard reference for agent memory design, which in turn imports Endel Tulving’s distinction between episodic and semantic memory from 1972. The mapping onto operating businesses is ours, developed from the deployments we have actually run in Alaska.
Keep reading
Small models, close to the data
Why the frontier is moving toward models that run on hardware you own, and what that unlocks here.
Read →
Industry playbooks
The same architecture applied to fourteen Alaska industries, workflow by workflow.
Read →
Ten questions for any AI firm
How to tell, before you sign, whether a vendor understands this.
Read →