Hybrid Robustness Verification for Spatio-Temporal Neural Networks

Problem
The paper addresses the gap in formal robustness verification for spatio-temporal neural networks, particularly in safety-critical applications such as autonomous driving and medical imaging. Existing methods often rely on conservative approximations or are computationally prohibitive, failing to account for the structured nature of adversarial perturbations in video data. This work is a preprint and has not undergone peer review.

Method
The authors introduce Spatio-Temporal Bound Propagation (STBP), a verification framework specifically designed for 3D CNNs processing video and volumetric inputs. STBP computes an exact closed-form characterization of the first convolutional layer, which allows for the tightest bounds to be established. For subsequent layers, the framework employs scalable approximation methods to propagate these bounds. The method leverages realistic assumptions about adversarial perturbations, modeling them as spatio-temporal constraints where attackers can modify a subset of frames or patches within consecutive frames. The framework is evaluated on three benchmarks: UCF-101 for action recognition, Udacity for autonomous driving, and MedMNIST for medical imaging.

Results
STBP achieves a 1.7x increase in certified robust accuracy compared to existing verification-based approaches under identical perturbation budgets. The paper demonstrates that by modeling adversarial perturbations with realistic constraints, the framework provides stronger robustness guarantees while maintaining improved scalability. The results indicate that STBP outperforms traditional lp-norm perturbation methods, which do not account for the structured nature of adversarial attacks in video data.

Limitations
The authors acknowledge that while STBP provides tighter bounds and improved scalability, the reliance on approximations in later layers may introduce some degree of conservativeness in the overall robustness guarantees. Additionally, the framework’s performance may vary depending on the specific architecture of the neural networks being verified. The paper does not address the potential computational overhead associated with the exact closed-form computation for the first layer, which may limit its applicability in extremely large networks.

Why it matters
This work has significant implications for the deployment of AI in safety-critical systems, where formal robustness guarantees are essential. By providing a more accurate and scalable verification method, STBP can enhance the reliability of AI models in real-world applications, particularly in domains where safety is paramount. The introduction of ST-Bench as a verification benchmark further facilitates systematic evaluation of verifiable robustness in spatio-temporal networks, paving the way for future research in this area. This is crucial for advancing the state of the art in AI safety, as highlighted in the context of ongoing discussions in the field, as published in arXiv cs.AI.