ReSET: Accurate Latency-Critical NVFP4 Reasoning via Step-Aware Temperature Scaling

Problem
This work addresses the limitations of applying NVFP4 (NVIDIA’s Variable-Precision Floating Point 4) quantization to large reasoning models (LRMs), which suffer from degraded reasoning accuracy and suboptimal latency during small-batch autoregressive decoding. The authors highlight that existing literature does not adequately tackle the challenges posed by quantization-induced errors in token-level sampling and the inefficiencies of NVFP4 kernels in latency-critical scenarios. This paper is a preprint and has not undergone peer review.

Method
The core contribution is the ReSET (Reasoning Step Entropy-based Temperature scaling) method, which employs a novel temperature-scaling mechanism that adapts based on both token-level and step-level entropy signals. This approach allows for online estimation of uncertainty during reasoning, mitigating the adverse effects of quantization on sampling accuracy. Additionally, the authors introduce a CUDA-core small-$M$ NVFP4 kernel specifically designed for latency-sensitive autoregressive decoding, optimizing the execution of low-precision operations. The training compute details are not disclosed, but the method leverages existing NVFP4 infrastructure to enhance performance.

Results
ReSET demonstrates a significant improvement in reasoning accuracy, achieving up to a 2-point increase over the NVFP4 baseline across various reasoning benchmarks and model scales. The CUDA-core small-$M$ kernel provides a kernel-level speedup of up to 2.5× compared to NVFP4 vLLM and an approximate 2× end-to-end decoding speedup over BF16 precision. These results indicate that ReSET effectively balances the trade-off between accuracy and latency in low-precision inference.

Limitations
The authors acknowledge that while ReSET improves accuracy and latency, it may still be sensitive to the specific characteristics of the reasoning tasks and the underlying model architecture. They do not discuss potential limitations related to the generalizability of the method across different model sizes or types, nor do they address the computational overhead introduced by the entropy estimation process. Additionally, the performance gains are contingent on the effective implementation of the CUDA-core kernel, which may not be universally applicable across all hardware configurations.

Why it matters
The implications of this work are significant for the deployment of large reasoning models in latency-sensitive applications, such as real-time decision-making systems and interactive AI. By improving the efficiency of low-precision inference, ReSET paves the way for more practical applications of LRMs in resource-constrained environments. This research contributes to the ongoing discourse on optimizing model performance while maintaining accuracy, as highlighted in related works on quantization and model efficiency, and is available on arXiv.