On-Policy Delta Distillation

July 14th ~ July 21st
#117 Latest AI Research Explained Simply

🗞️ Industry News in 1 Line

1. ♥ 14k Thinking Machines has released Inkling, a new open-weights multimodal model designed for reasoning across text, image, and audio inputs. The model introduces a continuous thinking feature, which allows users to adjust the balance between computational cost and performance according to their specific needs. You can try it on Hugging Face.

   

2. ♥ 24k Alibaba has teased the upcoming open-weight release of Qwen3.8, a huge model with 2.4 trillion parameters. The weights are not released yet but, developers can access and evaluate the Qwen3.8-Max-Preview version through Alibaba’s Token Plan, Qoder, and QoderWork platforms.

   

3. ♥ 56k Kimi has introduced Kimi K3, a 2.8-trillion-parameter multimodal model designed with a 1-million-token context window for long-horizon agentic workflows. To optimize performance, the architecture uses Kimi Delta Attention to accelerate decoding in long contexts alongside Attention Residuals to improve training throughput. You can try it on GitHub or Hugging Face.

   

Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes

Le et al. [Google, Google DeepMind, Stony Brook University]

♥ 184   Image Generation  

Most current AI models generate text and images either in separate turns or in parallel tracks that do not communicate during the actual creation step. Because these models cannot take back a decision once it is made, a small mistake early on can lead to contradictions, where the generated text describes one thing but the accompanying image shows something else entirely.

To solve this, researchers introduced a framework called Self-Correcting Coupled Markov Jump Processes, alongside a practical, single-pass sampler named CO2Jump. Instead of generating text and images in isolation, this system allows both paths to actively negotiate with each other at every step of the generation process.

By looking at how confident one side is, and mapping that confidence to the other side using internal attention signals, the system balances the two modalities dynamically. If the image generator is struggling to render a complex scene, it can pause and rely on the text's plan; conversely, if the text is uncertain, it looks to the emerging image for guidance.

The most significant breakthrough of this approach is its ability to self-correct. If the system detects a contradiction between what it is writing and what it is drawing, it uses a remasking technique to essentially erase its recent commitments and try again.

Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers

Fan et al. [Microsoft Research, ETH Zürich, KRAFTON, Ludo Robotics]

♥ 875   Looped Transformer   bycloud’s pick  

As AI models grow larger, their internal reasoning drifts off course and loses accuracy compared to typing things out. Current methods process these hidden steps one by one, which fails to solve the speed bottleneck.

To solve this, researchers developed a training recipe called LOTUS. Instead of processing thoughts one after the other, LOTUS uses a "looped" model architecture. It sets up a series of placeholder slots and refines them all in parallel, passing them through the same network weights over several loops.

To keep these hidden thoughts from drifting into nonsense, the researchers applied a clever training method: they supervised the model's internal layers directly, aligning them with the actual text-based reasoning steps the model should be taking.

This approach yielded highly promising results at the three-billion-parameter scale. LOTUS successfully matched the accuracy of traditional step-by-step text generation on math benchmarks, and even proved more adaptable on unfamiliar problems.

By replacing sequential typing with parallel internal loops, it slashed the time spent in the "thinking" phase by 2.5 times for compact math expressions, and by nearly seven times when dealing with longer natural language explanations.

When they translated its final internal states back into readable text, they found they could easily recover the correct step-by-step reasoning. Even more surprising, these internal states also held alternative, equally valid pathways to the correct answer that the model had never been explicitly trained on.

pykci: A Compact Urban Knowledge Graph for Semantic and Spatial Queries using LLMs

Nguyen et al. [HafenCity University Hamburg]

♥ 430   LLM Harness  

This paper introduces a new open-source framework that transforms complex 3D city datasets into a compact urban knowledge graph. Instead of mirroring every redundant layer of raw structural hierarchy, the system absorbs unnecessary intermediate wrappers into direct relationships between city objects, boundary surfaces, and geometry.

This design preserves complete semantic and geometric details and allows complete, lossless reconstruction of the original files, while drastically shrinking database size and shortening query traversal paths. If it is combined with dedicated spatial indexing, the system makes searching across entire metropolitan datasets significantly faster and lighter.

This architecture allows LLMs to serve as plain-language query interfaces. Rather than requiring users to write technical database code, the system translates simple conversational questions into precise graph searches.

Because the graph structure is so streamlined, even smaller open-weight AI models can run locally on private municipal servers to answer complex spatial and environmental queries reliably.

Urban planners and citizens alike can now simply ask human questions, such as locating dark rooftops suitable for greening or identifying optimal buildings for solar panel retrofits, and receive instant, auditable answers grounded in exact city data.

On-Policy Delta Distillation

Heo et al. [NAVER AI Lab]

♥ 373   LLM Training  

Smaller "student" models learn by directly mimicking a larger "teacher" model's overall output behavior. However, this approach meant students often copied the teacher's generic writing styles and conversational habits rather than the actual logical steps required to solve tough mathematical or scientific problems.

To address this, researchers developed On-Policy Delta Distillation, or OPD2. Instead of forcing the student to copy everything the teacher does, this method isolates a "delta signal" which measures the difference between the advanced, reasoning-tuned teacher and that same teacher's raw base model before it received specialized reasoning training.

By focusing on this difference, the student receives a highly concentrated learning signal that highlights the logical upgrades the teacher underwent. This filters out unnecessary stylistic noise, guiding the student to focus on explicit reasoning connectives and logical pathways.

To keep the training process stable and practical, the team introduced centering techniques to balance the learning signals and added an agreement condition. This condition acts as a safeguard and ensures the student only updates its knowledge when the delta signal aligns with the teacher's actual guidance, preventing the student from drifting off course.