Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks

Problem
This paper addresses the significant accuracy collapse observed in high-sparsity regimes following one-shot magnitude pruning of neural networks, particularly in vision tasks. The authors highlight a gap in existing literature regarding the granularity of post-pruning repair methods, which typically apply a uniform correction across entire layers. This approach fails to account for the varying degrees of damage across individual channels, leading to ineffective recovery of the network’s performance. The work is presented as a preprint and has not yet undergone peer review.

Method
The authors propose a novel method called Adaptive Signal Resuscitation (ASR), which implements a channel-wise repair strategy that aligns the granularity of the correction with the granularity of the damage incurred during pruning. ASR operates without retraining, requiring only forward passes on a small calibration dataset. The method estimates a variance-matching correction for each output channel, which is then stabilized using a data-driven shrinkage rule. This shrinkage suppresses corrections for channels exhibiting weak post-pruning signals while preserving corrections for healthier channels. ASR is applied prior to BatchNorm recalibration, enhancing the overall effectiveness of the repair process.

Results
ASR demonstrates substantial improvements over existing layer-wise repair methods across multiple datasets and architectures. Specifically, on ResNet-50 at 90% sparsity, ASR achieves a top-1 accuracy of 55.6% on CIFAR-10, significantly outperforming layer-wise repair (41.0%) and BatchNorm-only recalibration (28.0%). The authors report consistent gains across three datasets and four convolutional architectures, particularly in high-sparsity scenarios. Ablation studies indicate that naive channel-wise variance matching alone is inadequate, and the incorporation of shrinkage is crucial for stabilizing post-pruning repair.

Limitations
The authors acknowledge that ASR’s reliance on a calibration set may limit its applicability in scenarios where such data is not readily available. Additionally, while the method shows promise in high-sparsity regimes, its performance in low-sparsity settings remains unexplored. The paper does not address potential computational overhead associated with the forward passes required for calibration, nor does it evaluate the method’s effectiveness across a broader range of architectures beyond the four tested.

Why it matters
The implications of this work are significant for the field of model compression and efficient neural network deployment. By providing a more nuanced approach to post-pruning repair, ASR enables the recovery of performance in highly sparse networks, which is critical for deploying lightweight models in resource-constrained environments. This research could pave the way for further advancements in adaptive pruning techniques and inspire new methodologies that consider channel-level characteristics in network optimization.

Authors: Qishi Zhan, Ziheng Chen, Minxuan Hu
Source: arXiv:2605.21426
URL: https://arxiv.org/abs/2605.21426v1