In partnership with
Aug 4th ~ Aug 11th
#120 Latest AI Research Explained Simply
🗞️ Industry News in 1 Line
♥ 7.9k Prime Intellect has introduced Prime Agent, an open-source self-improving Recursive LM framework that treats context as a variable by using a IPython kernel to manage execution state outside the active context window. This architecture allows LLMs to program over history, launch sub-agents, and achieve strong performance gains on complex, long-horizon tasks such as building hardware emulators and solving game environments. You can explore the codebase on GitHub.
♥ 4.2k Liquid AI has released LFM2.5-2.6B, an open-weight, 2.6-billion parameter hybrid model optimized for local, on-device agentic workloads with zero data exfiltration. It supports a 128K context window and surpasses several larger models on benchmarks like ToolSandbox and IFStruct. You can access the weights on Hugging Face.
♥ 6.1k Cursor has open-sourced Mixture-of-Kittens, which is a deterministic Mixture-of-Experts (MoE) training megakernel designed for NVIDIA NVL72 architectures. By fusing all MoE communication and computation into a single kernel without requiring CPU-GPU synchronization, MoK achieves up to a 2.37x speedup over top public baselines and increased Cursor's internal training throughput by 1.41x. You can inspect the implementation and try it out on GitHub.
Domain Names + Web and Email Hosting Done Right
Tired of overpaying GoDaddy or Namecheap? There's a smarter option with Porkbun!
Porkbun is the domain registrar trusted by creators, developers, entrepreneurs, and anyone who wants low prices without the nonsense or the forced upsells.
Here's why so many people are making the move to the Bun:
• Most domains sold at cost
• Low, transparent registration and renewal pricing
• Free features like WHOIS privacy and SSL certificates
• Powerful web and email hosting options
• Real human support 24/7, 365 days a year
• Named the #1 domain registrar by Forbes Advisor and USA Today
Whether you're launching a business, building a personal brand, starting a side project, or creating your very first website, Porkbun makes the whole thing easy.
Leanstral
Mistral AI
♥ 372 LLM
Leanstral is an open-source model that can tackle advanced mathematics and catch hidden software bugs at a fraction of the cost, using the exact same tools human software engineers rely on every day.
Most top-performing mathematical provers rely on rigid, highly specialized search workflows that are disconnected from typical software tools. This custom scaffolding makes scaling computationally expensive and prevents developers from seamlessly using formal verification in real-world code repositories.
To solve this, researchers introduced Leanstral, a model that operates directly as a general-purpose software assistant rather than a specialized proof engine. Built with 119 billion total parameters, while activating just 6 billion for any given calculation.
It interacts with code much like a human developer: editing files, reading compiler feedback, inspecting language server diagnostics, and summarizing its history when working through long, complex tasks. Instead of requiring complex search algorithms, the model steadily improves its performance simply by taking more time to iterate through problems.
This straightforward approach achieved breakthrough results across formal mathematics and practical engineering. Leanstral surpassed or rivaled far larger proprietary systems on elite competition benchmarks like PutnamBench while running at a fraction of the cost.
Additionally, an automated pipeline powered by Leanstral successfully verified complex data structures and discovered previously unknown bugs in real-world open-source software.
Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes
Han et al. [FAIR, Reality Labs, University of Oxford]
♥ 760 Pre-Training bycloud’s pick
We build unified AI models by patching visual capabilities onto pre-trained text models. Without a clear blueprint, building unified multimodal systems relies heavily on guesswork rather than principled design.
To solve this, researchers systematically explored how language, visual understanding, and visual generation transfer knowledge among themselves during pretraining.
Overview of the synthetic CLEVR testbed.
They discovered a stark asymmetry in knowledge flow: text acts as a universal catalyst that boosts all visual tasks, and visual understanding provides strong guidance for generating images. While generating images does not automatically grant immediate improvements to text processing, it does something deeper.
Concept recovery via fine-tuning.
Learning to construct raw images forces the model to map dense geometric and spatial details, creating rich latent representations that dramatically accelerate how quickly the model learns complex visual reasoning later during fine-tuning.
Unifying text and visual streams from the very start of training is essential to prevent "vision laziness", a phenomenon where late-integrated models rely too heavily on language shortcuts instead of properly developing native visual processing.
Impact of vision encoder designs on modality synergy
On-Policy Self-Distillation without Any Supervision
Li et al. [UCSan Diego, Georgia Institute of Technology, University of Maryland, ByteDance]
♥ 511 Post-Training
Existing self-distillation techniques, where a model acts as its own teacher, depend on outside ground-truth answers to construct lessons. This reliance creates a major bottleneck, which limits a model's ability to scale up and learn.
Comparison between OPSD / SDFT with ground-truth solution or ICLs
To solve this, researchers introduced Unsupervised On-Policy Self-Distillation (U-OPSD), a technique that allows a model to teach itself using only its internal consensus. The system prompts the model to generate multiple solution attempts for a single problem and calculates a majority vote among the final answers.
When a strong consensus forms, the best agreeing attempt becomes a self-generated benchmark solution. The model then looks closely at its own minority attempts that arrived at wrong answers and uses the consensus solution to guide itself token by token.
Overview of Unsupervised On-policy Self-Distillation
This approach generates remarkable progress without relying on any external labels. Evaluated across challenging mathematical reasoning benchmarks, the unsupervised method consistently outperformed base models and even surpassed supervised techniques that had full access to ground-truth answer keys.
The Loss Does Not See the Basis, but Adam Does
Singh [Memorial University of Newfoundland]
♥ 195 LLM Optimization
Many popular AI training algorithms evaluate parameter coordinates individually along an arbitrary grid, which breaks an internal rotational symmetry called gauge equivariance. This coordinate dependence destroys the model's natural tendency to seek simple, low-rank solutions, causing it to settle on far more complex internal representations.
Equivariance structures the set of available optimizers under a fixed computational budget.
Researchers discovered that an optimizer’s ability to recover simple structures hinges directly on whether it preserves this mathematical gauge symmetry. Optimizers that respect the symmetry, such as standard gradient descent, Muon, Shampoo, or a modified "scalar" Adam, treat internal representations identically regardless of how their coordinate bases are rotated.
This symmetry allows the algorithm to follow a balanced spectral schedule, growing parameter directions sequentially and keeping the model focused on the true underlying signal.
As Adam’s preconditioner changes shape from coordinate-wise towards isotropic, the effective rank of the solution gracefully decreases towards that of standard gradient descent (GD).
By contrast, standard coordinate-wise algorithms break this symmetry on their very first step. In transformer attention heads, training two functionally identical models initialized in different coordinate frames causes standard Adam to immediately drift apart in its internal representations, whereas symmetry-preserving optimizers remain aligned at machine precision.
The researchers mathematically proved that memoryless update rules must operate as coordinate-independent preconditioners to preserve this low-rank preference.