Topic
fast
StreamKL Delivers up to 43× Speedup in Memory-Efficient Attention Distillation
Researchers propose StreamKL, a fused GPU primitive for Kullback-Leibler divergence in attention distillation. It eliminates quadratic memory materialization, enabling up to 43× and 14× speedups in forward and backward passes, and reduces extra HBM footprint to O(1).
SafeSpec: New Framework Boosts LLM Safety Without Sacrificing Inference Speed
Researchers propose SafeSpec, a safety-aware speculative inference framework that attaches a latent safety head to jointly evaluate semantic validity and safety in a single forward pass. On Qwen3-32B, it reduces attack success rates by 15% while preserving a 2.06x inference speedup on benign workloads, addressing the fundamental incompatibility between existing safety methods and speculative decoding.
Fast-dLLM++ Boosts Diffusion LLM Inference Up to 37% With Fréchet Profile Decoding
Researchers propose Fast-dLLM++, a training-free extension to Fast-dLLM that uses Fréchet profile decoding to select parallel token commit sets from the full confidence profile. Experiments on LLaDA-8B show up to 37% higher throughput at comparable accuracy on benchmarks including GSM8K, MATH, HumanEval, and MBPP.