Explorative Modeling: Third Pre-training Axis?

July 2th ~ Aug 4th
#119 Latest AI Research Explained Simply

🗞️ Industry News in 1 Line

1. ♥ 22k Alibaba's Qwen team has announced Qwen3.8-Max, a 2.4-trillion parameter model for long-horizon task planning, native multimodal feedback loops, and autonomous software development. The model is currently accessible via API, with open weights for both Qwen3.8-Max and the 27B variant scheduled for release next week.

   

2. ♥ 29k DeepSeek has launched the public beta for its updated version of DeepSeek-V4-Flash-0731 API. The updated endpoint retains the same model architecture while containing a huge jump in performance, introducing native support for the Responses API format, and full optimization for Codex integrations. You can review the documentation and test the new API on DeepSeek's developer portal.

   

Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation

Gladstone et al. [UIUC, Harvard]

♥ 2.4k   pre-training

Real-world data often contains many valid outcomes for a single input, standard training forces models to average these possibilities together, which leads to blurry outputs and compounding per-step errors.

To solve this, researchers introduced Explorative Modeling, this changes how generative models are trained rather than how they generate outputs. Instead of forcing a network to predict a single averaged answer during a training step, Explorative Models generate multiple candidate options, compare them to the target data, and update the network using only the best match.

This allows the system to commit entirely to clear, distinct possibilities rather than muddying them into an artificial average, unlocking true end-to-end learning where training and final generation operate identically.

The researchers discovered that adding exploration creates a powerful third axis for scaling AI capabilities, sitting alongside parameter size and dataset size. Across continuous and discrete tasks, including image, video, and language generation, increasing exploration monotonically improves output quality.

The approach dramatically boosts efficiency and it allows models to match baseline performance using 6.2 times less training data, 4.1 times fewer compute operations, and 47 percent fewer parameters.

Metis: Memory Foundation Model

Zhang et al. [MemTensor (Shanghai) Technology Co., Renmin University of China, National University of Singapore, Shanghai Jiao Tong University, Tongji University]

♥ 652   LLM Memories   bycloud’s pick  

AI agents manage memory using external text-retrieval systems. Because these external systems operate outside the model's primary neural network, they introduce frustrating delays and prevent the AI from optimizing its memory handling during training.

To solve this challenge, researchers introduced the concept of memory foundation models and developed a pioneering prototype named Metis. Rather than relying on outside text databases, Metis internalizes memory directly into dynamic parameters within its internal architecture.

The system uses specialized internal blocks that dynamically compress past conversation details straight into the model's latent state. As new instructions arrive, Metis autonomously updates, remembers, or forgets information during its standard forward computation, entirely eliminating the need for slow external lookup routines.

The researchers trained Metis through specialized mid-training tasks that taught the system to predict how past information should be stored and retrieved for future steps. During real-time use, the model's primary parameters stay frozen while its internal memory states adapt fluidly from one interaction to the next.

Weak-to-Strong On-Policy Distillation

Yu et al. [University of Maryland, Microsoft Research, MBZUAI]

♥ 353   Distillation  

Distillation uses a superior model to coach a weaker one, but this framework breaks down at the frontier where no stronger teacher exists, and building massive expert teachers from scratch is wildly expensive.

To overcome this wall, researchers introduced Weak-to-Strong On-Policy Distillation (W2S-OPD). This allows a powerful student model to keep improving using guidance from smaller, less capable models.

Instead of forcing the strong student to blindly copy a weaker teacher, which risks dragging its overall intelligence down, the new method isolates a pure "capability direction." It does this by taking a contrast pair of weak models, a slightly better "positive" model and a basic "negative" model, and calculating the difference between their output predictions.

Subtracting the negative model from the positive model cancels out their shared weaknesses, leaving behind only the mathematical signal representing the specific skill gap, such as refined reasoning or coding logic.

This isolated skill direction is added onto the strong student model’s own starting base, creating a temporary "proxy teacher" that remains close to the student's baseline while offering targeted improvements.

W2S-OPD consistently outperformed standard distillation and enabled student models to surpass the teachers guiding them. Interestingly, contrasting different pairs taught complementary skills: comparing models before and after reinforcement training boosted high-level reasoning frameworks, while contrasting models of different scales sharpened step-by-step execution.

LeRoPE: Learnable RoPE Frequencies Improve Language Modeling

Karypis et al. [UC San Diego]

♥ 348   LLM Positional Embeddings  

LLMs use a technique called Rotary Positional Encodings, or RoPE, to track the order of words by rotating mathematical vectors at fixed rates. These rotation rates are fixed human choices set before training starts, forcing models to process spatial positions using rigid rules rather than learning what works best.

To solve this, researchers introduced Learned RoPE, or LeRoPE, a lightweight modification that turns these fixed rotation rates into flexible parameters the model can tune during training.

By adding just 32 trainable numbers across the entire network, the system gains the ability to fine-tune how fast or slow each frequency rotates. The researchers discovered that as models train with LeRoPE, they automatically suppress unhelpful ultra-slow frequencies and naturally form a single "dominant" positional frequency.

LeRoPE consistently outperforms traditional fixed approaches, particularly on demanding context-retrieval tasks where models must find key facts hidden among multiple distracting details. RoPE requires 3.4 percent more training compute to match the performance level achieved by LeRoPE at the largest model size.