Topic
memory optimization
Artificial Intelligence #techniques#peak memory reduction
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
A new paper introduces four complementary techniques to reduce peak memory during LoRA fine-tuning of large language models on edge devices. Experiments on Llama-3.2 3B and Qwen-2.5 3B demonstrate up to 26x and 28x memory reduction, respectively, without sacrificing model quality.
Jun 20, 2026 1 source
Artificial Intelligence #kv cache#compression
PolyKV: Layer-Wise KV Cache Compression Boosts LLM Inference Efficiency by Up to 54.5%
PolyKV is a new framework for compressing the key-value cache in large language model inference. It selects a compression policy per transformer layer and allocates non-uniform cache budgets, outperforming uniform approaches. On LongBench tasks, PolyKV recovers 40%-54.5% of the performance gap between the strongest single-policy baseline and full KV cache.
Jun 16, 2026 1 source