Beyond Lipschitz: Data-Driven Robustness via Discrete Modulus of Continuity

Problem
This paper addresses the limitations of traditional Lipschitz continuity as a measure of robustness in neural networks. The authors argue that Lipschitz constants can be either too coarse or overly restrictive, failing to capture the nuanced, data-dependent behaviors of models. This work is presented as a preprint and has not yet undergone peer review.

Method
The authors introduce a framework based on the discrete modulus of continuity (DMOC), which serves as a non-linear generalization of Lipschitz continuity. DMOC is architecture-agnostic and evaluates the regularity of a model relative to the data distribution, rather than relying on model internals. The paper establishes convergence results for DMOC-induced seminorms, providing explicit data-driven rates based on the separation distance. A scalable minibatch algorithm is proposed to reduce the quadratic computational cost associated with exact DMOC computation, making it feasible for large datasets like ImageNet. The method allows for efficient assessment of model robustness without requiring extensive model-specific adjustments.

Results
Empirical evaluations demonstrate that DMOC effectively distinguishes between trained and untrained networks, identifies underfitting and overfitting regimes, and provides tight Lipschitz estimates. The authors compare DMOC against state-of-the-art methods such as ECLipsE and ECLipsE-fast, showing that DMOC yields comparable results while being more flexible and data-driven. Specific performance metrics are not disclosed in the abstract, but the authors claim that DMOC serves as a robust diagnostic tool across various architectures.

Limitations
The authors acknowledge that while DMOC provides a more nuanced measure of robustness, it may still be sensitive to the choice of data distribution and may not generalize well across all types of neural architectures. Additionally, the computational efficiency of the proposed minibatch algorithm, while improved, may still pose challenges for extremely large datasets or real-time applications. The paper does not address potential limitations related to the interpretability of DMOC results or the implications of its application in adversarial settings.

Why it matters
This work has significant implications for the field of robust machine learning, as it shifts the focus from model-centric evaluations to data-driven assessments of robustness. By providing a more refined measure of regularity, DMOC could enhance the understanding of model behavior in various training regimes and improve the robustness of neural networks against adversarial attacks. The architecture-agnostic nature of DMOC also suggests that it could be widely applicable across different types of models, potentially influencing future research on robustness metrics and their integration into model training and evaluation processes.

Authors: Jürgen Dölz, Michael Multerer, Michele Palma
Source: arXiv:2605.28729
URL: https://arxiv.org/abs/2605.28729v1