Topic
cache
Artificial Intelligence #llm#ai
OBCache Prunes KV Cache for Efficient Long-Context LLM Inference with Output-Aware Scoring
A new method called Optimal Brain Cache (OBCache) treats key-value cache eviction as a layer-wise structured pruning problem. By measuring token saliency through perturbation in attention outputs, OBCache outperforms heuristic-based approaches on LLaMA and Qwen models, consistently improving long-context accuracy according to the paper.
Jun 16, 2026 1 source
Artificial Intelligence #llm#inference
New VeriAttn Technique Accelerates Verifiable LLM Inference on TEE-GPU Systems
Researchers propose VeriAttn, a communication-efficient TEE-GPU attention mechanism for verifiable LLM inference. By offloading attention computations to the GPU while the TEE performs verification, VeriAttn achieves 2.60-3.38x acceleration for prefill and 3.86-5.42x for decoding over the TSDP baseline on Intel TDX.
Jun 16, 2026 2 sources