Tag: quantization
-
Google’s TurboQuant Cuts LLM Memory 6x with Zero Accuracy Loss
On March 25, 2026, Google Research introduced TurboQuant — a new compression algorithm that reduces LLM key-value (KV) cache memory by 6x and delivers up to 8x inference speedup on NVIDIA H100 GPUs, all with zero accuracy loss. The algorithm, set to be presented at ICLR 2026, could reshape how organizations deploy large language models […]
-
Advancing LLM Training: Introducing NVFP4 for Efficient Pretraining
Key points The study presents NVFP4, NVIDIA 4-bit floating point precision, as a way to improve large language model training efficiency without compromising model quality. The article states that FP4 training poses critical challenges including training stability for large-scale models, convergence issues with long token sequences, and implementation complexity in maintaining accuracy. The NVFP4 framework […]