Study finds AI models store memories and logic in different neural regions

Recent research from the AI startup Goodfire.ai has unveiled significant insights into the inner workings of AI language models, such as GPT-5. The study identifies two key functions within these models: memorization, which allows them to recall exact text, and reasoning, which enables them to tackle new problems using established principles. The findings, detailed in a preprint paper published in late October, suggest that these two capabilities operate through entirely different neural pathways in the architecture of the models. Remarkably, the separation between these functions is distinctly clean. Upon removing the memorization pathways, the researchers observed a staggering 97 percent decline in the models' ability to recite training data verbatim, while their logical reasoning capabilities remained largely unaffected. For instance, in the OLMo-7B model developed by the Allen Institute for AI, the bottom half of the weight components demonstrated a 23 percent increase in activation for memorized data, whereas the top 10 percent showed a 26 percent increase for general, non-memorized text. This clear demarcation allowed the researchers to effectively eliminate memorization without compromising other functionalities. Perhaps the most intriguing outcome of the study is the revelation that arithmetic tasks utilize the same neural pathways as memorization instead of logical reasoning. When the memorization circuits were disabled, performance on mathematical operations dropped dramatically to 66 percent, while the models maintained their logical task performance. This highlights a potential reason behind the difficulties AI language models face with mathematical computations; they appear to rely on a limited memorization of arithmetic rather than actually processing the calculations, akin to a student who has memorized multiplication tables without grasping the underlying principles. It’s important to note that the concept of reasoning in AI encompasses a range of abilities that may not align perfectly with human reasoning. The logical reasoning that persisted after the removal of memory in this study included tasks like assessing true/false statements and applying if-then rules, which primarily involve leveraging learned patterns on new data. This contrasts with the more complex “mathematical reasoning” required for proofs or innovative problem-solving, areas where current AI models continue to struggle, even with intact pattern-matching skills.