You Only Need 1 Layer for RLVR?

July 1st ~ July 7th
#115 Latest AI Research Explained Simply

🗞️ Industry News in 1 Line

1. ♥ 2.2k Tencent has launched Hy3, a new 295-billion parameter Mixture of Experts model optimized for productivity and agentic workflows. This model is released under the Apache 2.0 license, and it provides an accessible and cost-effective option for commercial applications. You can try it on Hugging Face.

   

2. ♥ 1.4k Mistral has released Leanstral 1.5 (Le Chaton L∃∀N), a formal-reasoning model designed for mathematical theorem proving and code verification using the Lean language. The model demonstrates efficient test-time scaling on the PutnamBench benchmark and includes a translation pipeline to assist with verifying Rust code. You can try it on GitHub or Hugging Face.

   

3. ♥ 3.7k Meituan has introduced LongCat-2.0, a 1.6-trillion parameter Mixture of Experts model designed for agentic coding with a 1-million token context window. The model uses sparse attention and dynamic expert routing to optimize computational efficiency during multi-file codebase analysis and complex reasoning tasks.

   

Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

Zhang et al. [University of Minnesota, Peking University, Amazon]

♥ 22k   LLM Training  

We train AI models by updating every single layer of the network uniformly, assuming that every part must adapt together to make progress. However, this approach has kept us in the dark about where the learning actually takes place inside the model.

To investigate this, researchers trained individual layers of various language models while freezing the rest of the network. They discovered that vast majority of RL gains are concentrated in just a small handful of layers located right in the middle of the model.

Training just one of these middle layers could recover almost all (and sometimes even exceed) the performance improvements achieved by updating the entire model. Meanwhile, the layers at the very beginning and the very end of the network contributed almost nothing to the final results.

This middle-focused behavior remained consistent across different model sizes, training datasets, and diverse tasks like mathematics, coding, and multi-step decision-making.

DOPD: Dual On-policy Distillation

Yu et al. [NUS, MMLab, PKU, Explore Academy]

♥ 424   Distillation   bycloud’s pick  

Training smaller AI models is a delicate art, especially when we try to assist the "teacher" model with extra hints that the student will not have access to in the real world.

When teacher models rely on privileged clues to guide a student, it often creates a "privilege illusion" where the student tries to copy the teacher’s final outputs without actually understanding the underlying logic. This information mismatch confuses the student, leading to unstable training and a decline in its independent reasoning abilities.

To address this challenge, researchers developed a framework called Dual On-policy Distillation, or DOPD. The system works by analyzing what they call the "privilege advantage gap". This metric compares the prediction confidence of the teacher and the student when both are given the same privileged hints. By evaluating this gap on a token-by-token basis, the framework can pinpoint exactly which parts of a response represent genuine, transferable reasoning skills and which parts are merely shortcuts derived from the extra clues.

Once these learning scenarios are identified, the system routes different teaching strategies to different tokens. For words where the teacher demonstrates a clear capability advantage, the student receives strong, full-vocabulary guidance to absorb that deeper logic.

On the other hand, tokens that are either highly obvious or too uncertain, the system dials back the teacher's influence and instead uses the student's own internal consistency as a stabilizing guide.

Orca: The World is in Your Mind

Kim et al. [Orca Team, Beijing Academy of Artificial Intelligence]

♥ 430   World Models  

Most of today's AI systems are highly specialized, focusing strictly on predicting the next word, the next video frame, or the next robotic motion in isolation. This fragmented approach prevents them from developing a unified, common-sense understanding of physical cause and effect. To address this, researchers developed Orca, a foundational world model designed to learn a unified, hidden representation of physical transitions.

Orca builds this understanding using two complementary learning pathways. The first is an "unconscious" learning process, where the model watches thousands of hours of continuous, unlabeled video to quietly absorb the dense, natural laws of how objects move and scenes change.

The second is a "conscious" pathway, which pairs video segments with language annotations to help the model connect these physical transitions to explicit human concepts, causal relationships, and task instructions.

To prove that this learned internal map is actually useful, the researchers froze Orca’s core architecture and attached lightweight, trainable decoders to translate its hidden states into text, predicted future images, and physical robot actions.

AdaJEPA: An Adaptive Latent World Model

Karan and Du [Harvard University]

♥ 855   LLM World Models  

Most AI "world models" are kept frozen after their initial training. When these models encounter minor real-world variations, such as unexpected lighting, shifting friction, or novel obstacles, their predictions quickly drift, causing their entire planning process to fail.

To solve this, researchers developed AdaJEPA, an adaptive framework that introduces a continuous loop of planning, acting, adapting, and replanning. Instead of blindly relying on static pre-trained assumptions, this system executes a planned action, observes the actual outcome, and immediately uses that real-world feedback as a self-supervised signal to recalibrate its internal model.

By performing lightweight, single-step updates to just a small subset of its parameters right before the next plan is made, the system corrects its course dynamically without needing any extra expert guidance.

The researchers discovered that this real-time tuning gives steady improvements in planning success across a variety of challenging tasks, including navigating unfamiliar mazes and manipulating objects with unseen shapes or altered physics.

Program-as-Weights: A Programming Paradigm for Fuzzy Functions

Zhang et al. [Meta, UT Austin, UCL, UC Berkeley, Harvard University, Periodic Labs]

♥ 805   LLM Weights  

Many common programming tasks, such as filtering system logs, repairing malformed data, or sorting search results by intent, are too subjective for traditional, rigid code, but calling giant cloud-based models for every minor task is expensive, slow, and difficult to keep consistent.

To solve this, researchers have designed a system called Program-as-Weights, or PAW. Instead of querying a massive cloud artificial intelligence every time a user inputs data, PAW uses a specialized compiler to translate a developer’s natural-language instructions into a tiny, reusable file of custom neural network weights.

This small file acts as a specialized adapter that can be instantly loaded onto a lightweight, frozen interpreter model already installed on the user's local device. In essence, the heavy lifting of understanding the task happens only once during compilation, generating a compact, private tool that can be called repeatedly offline.

To make this possible, the researchers trained their compiler on a massive new dataset containing ten million examples across hundreds of text-processing categories. They found that a tiny local interpreter running these compiled programs outperformed a model over fifty times its size that was prompted in the traditional way.

This model was executed on standard consumer laptops, and it achieved fast execution speeds while requiring only about one-fiftieth of the memory typically needed by larger systems.