Bliss Reading Group - June 22

Join us for a deep dive into the innovations in long-context LLMs! 🌟 This week, Justus Westerhoff will present:

Token Sparse Attention 🤖

Explore how this approach tackles the quadratic scaling of attention costs with sequence length. As more tokens are added, inference can slow disproportionately. Traditional solutions often rely on fixed sparsity or eliminate tokens prematurely, potentially missing crucial information.

Jo et al. (2026) introduce a paradigm shift by suggesting that the relevance of tokens varies from layer to layer. Instead of discarding tokens, Token Sparse Attention selectively compresses each head's Q/K/V into a relevant subset for attention computations—recompressing results back into the entire sequence. 🌀

This innovative design integrates seamlessly with Flash Attention and existing sparse-attention kernels, achieving an impressive up to 3.23× attention speedup at 128K context with minimal accuracy loss (under 1%).

We'll discuss vital questions such as:

• Is reversibility the missing link?
• Can we maximize efficiency with smarter token eviction strategies?
• With context windows potentially expanding to millions of tokens, will token-selection be sufficient, or is a shift from softmax attention required?

Let's engage in a stimulating discussion! 💬

Location: Merantix AI Campus
Ticket Info: Get your ticket here! 🎟️