Balestriero and LeCun [Brown University, New York University, AMI Labs]

♥ 2.8k JEPA

AI models trained on separate senses, such as text and images, eventually converge toward an identical understanding of reality. This paper introduced a formal theorem by analyzing Joint-Embedding Predictive Architectures. The researchers discovered that models trained independently across different modalities provably learn the exact same internal map of the world, differing only by a single geometric rotation.

Standard language models rely on next-token prediction, which is a "bad view" that merely links fragments of a single document. This localized framing squanders model capacity on superficial syntax rather than universal concepts, explaining why conventional language models align with visual systems only through sheer, inefficient scale.

Training a model with good views proved 42 times more sample-efficient than next-token prediction at achieving cross-modal alignment. Moreover, a 109-million-parameter text model trained with good views aligned with leading vision systems far better than 72-billion-parameter language models trained on over a hundred times more data.

Simply calculating a geometric rotation between these independently trained encoders enabled them to caption images and classify objects without requiring networks trained on paired image-text datasets.

Song et al. [Carnegie Mellon University, Johns Hopkins University, Aurora Innovation]

♥ 1.5K LLM Training bycloud’s pick

When you train AI models, different systems are built with rigid, isolated stages which absorb vast amounts of text during pretraining, studying demonstrations during fine-tuning, and refining behavior through trial-and-error reinforcement learning.

Researchers use RL as maximum-likelihood estimation and they brought every phase of training under a single likelihood-based objective. Under this unified perspective, the learning algorithm remains constant from start to finish. The entire lifecycle instead becomes a fluid question of data mixture design: deciding which data sources to introduce, in what proportions, and at what points in time. Traditional staged training is simply an extreme, rigid variation where one data source abruptly cuts off before the next begins.

To make this practical, the researchers built an asynchronous training system that feeds both static text and reward-weighted model generations into a shared data buffer. A single trainer pulls from this mixed pool, applying a consistent weighted loss across all examples while periodically synchronizing its updated weights back to the text generator.

In controlled and real-world trials, this unified approach achieved higher accuracy and broader task coverage within the same compute budget. Even better, it protected general capabilities from degrading and provided a much stronger foundation for future continual learning, pointing toward a simpler, more cohesive science for building artificial intelligence.

Cao et al. [University of California]

♥ 492 LLM Agent

Automated software assistants struggle when optimizing complex hardware code. These systems hit performance ceilings because they rely on rigid, preplanned workflows, which lead them to abandon difficult but superior designs in favor of easy, slower alternatives whenever they encounter stubborn bugs.

To solve this, researchers introduced Evolving Agent Teams, an approach where an automated programming team continuously reshapes its own organization on the fly. A central supervisor monitors ongoing work, diagnoses why progress has stalled, and invents specialized roles tailored to the current challenge.

For example, if workers cut corners under pressure, the supervisor deploys a disciplinarian to strictly enforce coding standards. If a tricky bug halts the workflow, it assigns an isolated debugger to a parallel branch so other teammates can keep optimizing without interruption.

When AI agents were asked to accelerate intensive GPU programs, the adaptable teams consistently outperformed both standalone coding models and fixed groups of specialists.

On a demanding decode task for modern language models, the evolving team produced specialized code that ran twenty percent faster than expert handwritten production baselines, beating them across dozens of real-world workloads.

The framework achieved similar breakthroughs on linear recurrent networks, which specialized multi-GPU communication, mobile chips, and even simulated robot manipulation.

Ekbote and Du [FAIR at Meta, MIT]

♥ 350 Loop Transformer

Building good LLMs requires packing billions of new parameters into computer memory, which rapidly exhausts hardware limits. Looping offers an alternative approach by passing data through the exact same network layers multiple times, effectively creating a deeper architecture without storing extra weights.

However, endlessly repeating these computations can quickly inflate training costs and dramatically slow down real-time text generation because each loop must execute sequentially before the next can begin.

Researchers mapped these trade-offs across models up to one billion parameters. They discovered that repeating the entire network delivers the highest accuracy when parameter storage is the primary constraint, whereas looping only the middle layers provides the best return when training compute is limited.

They also found that simply rerunning identical weights eventually causes performance gains to stall. To overcome this plateau, the team introduced tiny, loop-specific low-rank adapters. This simple adjustment allows the network to learn slightly specialized transformations at deeper stages, unlocking higher quality with almost no increase in memory.

To solve the inference bottleneck where loops run one after another, the team investigated parallel architectures that transfer states across neighboring words, enabling multiple loops to run concurrently.

By separately normalizing the current word embedding and the prior loop state, and blending them together through a lightweight gate, this design enables fast, parallel text generation while matching or even exceeding the quality of standard sequential looping.

Teoh [MIT]

♥ 1.3K LLM Inference

Teaching LLMs to predict several upcoming words rather than just one accelerates text generation and improves multi-step reasoning. However, standard multi-token training injects a loud, conflicting residual gradient into the network trunk, which actively interfers with the core learning process as models scale up.

Researchers discovered that the standard multi-token training update naturally splits into two distinct mathematical forces. One is a beneficial self-distillation signal that aligns future predictions with the primary next-token engine, while the other is a noisy residual signal that accounts for roughly eighty percent of the update. This residual suffers from an exceptionally poor signal-to-noise ratio and directly opposes the main learning objective.

Although researchers previously tried to isolate the clean distillation signal directly, doing so required massive amounts of memory to store huge vocabulary tables during training. To solve this, the team developed an gradient-recovery trick which requires essentially a single line of code. By routing gradients through the network's existing mathematical outputs, their approach, called Soft-MTP, cancels out the noisy residual and delivers the pure distillation signal without extra computational or memory overhead.

The results show that stripping away this noise provides a clear, reliable advantage. In tests, Soft-MTP consistently lowered prediction errors, boosted text-generation speeds during speculative decoding, and improved downstream zero-shot accuracy.