🚨This week’s top AI/ML research papers - Oct 26th

🚨This week’s top AI/ML research papers:

• Sparse Crosscoders

• Rethinking Softmax

• ZIP-FIT

• Mechanistic Unlearning

• Decomposing The Dark Matter of Sparse Autoencoders

• Automatically Interpreting Millions of Features in Large Language Models

• Breaking the Memory Barrier

• Can Knowledge Editing Really Correct Hallucinations?

• Framer: Interactive Frame Interpolation

• Beyond Position

• A Hitchhiker's Guide to Scaling Law Estimation

• Scaling up Masked Diffusion Models on Text

• Why Does the Effective Context Length of LLMs Fall Short?

• Scaling Diffusion Language Models via Adaptation from Autoregressive Models

• Improve Vision Language Model Chain-of-thought Reasoning

• PyramidDrop

• FrugalNeRF

• SAM2Long

• SeerAttention

• FiTv2

overview for each + authors' explanations ⬇️ 

Sparse Crosscoders for Cross-Layer Features and Model Diffing

Overview: 

This research introduces "sparse crosscoders," a tool that tracks shared features across layers in neural networks, simplifying feature analysis and model comparisons. 

Crosscoders support long-term feature tracking, streamline circuit analysis by removing redundant features, and detect unique model differences, aiding in fine-tuning and architecture studies. 

Early results show they outperform per-layer methods in capturing cross-layer structures, though with higher computational cost.

Blog:

https://transformer-circuits.pub/2024/crosscoders/index.html 

Author's Explanation:

---

Rethinking Softmax: Self-Attention with Polynomial Activations

Overview:

This paper reevaluates the role of softmax in attention mechanisms, proposing that its effectiveness is due to implicit regularization of the Frobenius norm of the attention matrix rather than probability distribution generation.

The study introduces polynomial activations as alternatives that similarly regularize the Frobenius norm, showing potential for use in attention-based architectures.

Experiments reveal that these polynomial activations perform on par or better than softmax in multiple computer vision and language tasks, indicating promising new directions for attention mechanisms.

Paper:

https://arxiv.org/abs/2410.18613

---

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment

Overview:

ZIP-FIT introduces a data selection framework using gzip compression to evaluate alignment between training data and target task distribution, optimizing LLM performance for tasks like Autoformalization and Python code generation.

It significantly outperforms established methods such as DSIR and D4 by reducing cross-entropy loss and improving selection speed, showing efficiency improvements up to 85.1% and 65.8% respectively.

The approach highlights that smaller, well-aligned datasets can surpass larger, less targeted ones, emphasizing the importance of quality data over quantity.

The research underscores the efficacy of compression-based task alignment in enhancing domain adaptation and model learning efficiency.

Paper:

https://arxiv.org/abs/2410.18194

Author's Explanation:

---

Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization

Overview:

This paper explores the enhancement of knowledge editing and unlearning in LLMs through mechanistic interpretability, focusing on identifying specific model components (circuits) linked to particular mechanisms.

It finds significant differences in robustness when training components localized by various methods, especially distinguishing those preserving outputs from those identifying mechanisms with predictable intermediates.

Localizing edits to components linked with lookup-table mechanisms for factual recall enhances robustness across input/output formats and reduces side effects, outperforming baselines on datasets like sports facts and CounterFact.

Additionally, certain localized edits more effectively disrupt latent knowledge, making unlearning more resilient to attacks.

Paper:

https://arxiv.org/abs/2410.12949

Author's Explanation:

---

Decomposing The Dark Matter of Sparse Autoencoders