Why Memorized Knowledge Fails to Generalize in LLM Finetuning

July 7th ~ July 14th
#116 Latest AI Research Explained Simply

🗞️ Industry News in 1 Line

1. ♥ 17k Cursor and SpaceXAI has jointly trained Grok 4.5, which is a Mixture-of-Experts model designed to handle complex, multi-domain tasks across software engineering, data science, and legal work. The development team combined reinforcement learning with trillions of interaction tokens to teach the model how to navigate environments, recover from mistakes, and verify its own results.

   

2. ♥ 4.9k Cognition has built SWE-1.7, trained on a Kimi K2.7 base model using a stabilized RL pipeline. The model integrates with the Devin harness to perform deeper codebase research before executing edits, and it operats at speeds of up to 1,000 tokens per second on Cerebras hardware.

   

3. ♥ 4.9k Meta released Muse Spark 1.1, which can orchestrate multi-agent systems and manages complex, multi-app computer-use workflows. The model processes audio and visual inputs while dynamically choosing between scripting and direct UI interactions to complete tasks.

   

4. ♥ 12k OpenAI introduced GPT-5.6 with "Ultra Mode", this feature coordinates parallel agents to accelerate complex workflows. The update improves the model's computer-use capabilities, allowing it to inspect rendered visual outputs and export structured documents directly into professional tools. You can try the new model on ChatGPT or integrate it via the OpenAI API.

   

Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning

Hou et al. [Tsinghua University/Z.AI]

♥ 836   RL   bycloud’s pick  

Traditional training pipelines are highly inefficient because they force the AI to generate a large group of attempts and wait for the absolute slowest one to finish before updating, wasting valuable computing power.

To solve this bottleneck, researchers have developed an asynchronous framework called Single-rollout Asynchronous Optimization, or SAO, which allows the AI to learn from each individual attempt the moment it is completed.

This approach moves away from group-based training to a streamlined "single-rollout" method. Because updating a model continuously can introduce chaotic training errors and instability, the researchers designed a double-sided clipping mechanism that filters out extreme, disruptive policy changes.

They also introduced a smarter "critic" component that evaluates how well the AI is doing. By updating this critic twice as fast as the main AI model and freezing its core attention layers during training, they prevented mathematical updates from spiraling out of control. They also created a specialized evaluation tool that bypasses messy external environmental feedback, allowing the AI to focus strictly on its own sequence of decisions.

During testing, this training method shows excellent stability and it runs smoothly for a thousand steps, consistently outperforming older group-based methods on demanding reasoning and software engineering benchmarks. Because it processes feedback one path at a time, SAO also proved highly capable of helping models adapt to dynamically changing environments in online simulations.

Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents

Wu et al. [Meta AI]

♥ 424   Memory Agents  

During long tasks, AI agents frequently suffer from "behavioral state decay," meaning they lose track of original instructions, repeat previously failed attempts, or forget crucial environmental constraints simply because those details get buried in a rapidly growing history.

To address this, researchers developed a dual-agent architecture that separates action from memory. Instead of modifying the primary "action agent" or endlessly expanding its context window, they introduced a dedicated, parallel "memory agent."

This assistant observes the ongoing workflow, updates a structured memory bank containing verified facts and past failures, and carefully decides whether to inject a highly targeted reminder before the next action is taken.

Additionally, the memory agent can choose to remain silent if no intervention is needed, preserving the main agent’s focus and preventing cognitive overload.

This intervention strategy significantly improved task success rates across complex command-line debugging and interactive customer service simulations. Interestingly, the researchers found that selective silence was a key factor in these gains; constantly flooding the main agent with the entire memory bank actually degraded performance.

Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning

Dai et al. [HKUST]

♥ 584   LLM Finetuning  

When we update a language model, it quickly learns to recall individual facts but often fails to use them in complex, multi-step reasoning. This disconnect is known as the "Knowing-Using Gap". It is frustrating because a model might easily state a new fact in isolation but fail to connect the dots when asked a follow-up question.

To understand why this happens, researchers designed an internal diagnostic method called self-patching. This technique allows them to copy the model's internal representation of a fact from one layer and paste it directly into another layer during processing.

Through this, they discovered that the problem is not a lack of memory. Instead, the model successfully stores the new information, but holds it in its outer layers, which are the very early and very late stages of processing. Meanwhile, the complex reasoning circuits reside in the middle layers. The model simply fails to route the new information to where the actual problem-solving happens, leaving the knowledge stranded.

By manually redirecting the information from the outer layers to the middle layers, the researchers unlocked the model's latent reasoning abilities. Recognizing that searching for these layers manually is impractical for everyday use, they also developed a straightforward, pre-set routing strategy.

Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity

Cabannes et al. [Meta FAIR, Inria Paris & ENS-PSL University, University of Tübingen]

♥   LLM RNNs  

AI architectures either require an ever-growing, unsustainable amount of computer memory to recall past details, or they compress information so tightly into a tiny, fixed-size memory that they quickly become forgetful.

To break this bottleneck, researchers developed a new architecture called Sparse Delta Memory, which dramatically expands an AI’s memory capacity without demanding more computing power.

Upgrading a model's memory requires modifying every single connection at once, which becomes incredibly expensive as memory sizes scale up. The new approach solves this by introducing a sparse addressing scheme. Instead of rewriting the entire system at every step, the model selectively reads from and writes to only the most relevant parts of a massive, explicit memory bank. This allows the system's memory capacity to scale to orders of magnitude higher while keeping the required processing power completely constant.

The researchers discovered that this elegant design allows models to successfully retrieve information and learn from contexts stretching up to one million tokens. Furthermore, by training the model to store useful pretraining knowledge in its initial memory state, it demonstrated a stronger grasp of common-sense reasoning tasks. When tested at scale, this approach achieved a lower training loss and slightly better accuracy than traditional, resource-heavy architectures.