Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark

Problem
This paper addresses the lack of a unified framework for evaluating weakly supervised anomaly detection (WSAD) methods, which have evolved in three distinct directions: incomplete, inexact, and inaccurate supervision. The authors highlight that existing research in these areas is often isolated, making it difficult to assess whether they tackle unique challenges or share fundamental mechanics. This work is presented as a preprint and has not yet undergone peer review.

Method
The core technical contribution is the introduction of WSADBench, a comprehensive benchmark designed to standardize the evaluation of 36 algorithms across four modalities. The benchmark systematically varies label quantity, granularity, and quality to assess the performance of both specialized WSAD methods and advanced tabular foundation models. The authors conducted over 700,000 experiments to derive insights into the performance boundaries of various approaches. WSADBench includes open-source code and datasets to facilitate reproducibility and further research in WSAD.

Results
The findings reveal four critical insights:

1. There are strong intrinsic correlations between the different weak supervision scenarios, challenging the notion of their isolation.
2. Specialized WSAD algorithms perform well only in extreme label-scarcity conditions but are outperformed by tabular foundation models and general classification methods as the amount of supervision increases or in out-of-distribution (OOD) scenarios.
3. The utility of unlabeled data is inconsistent across different settings, showing only marginal gains compared to label refinement.
4. Models demonstrate asymmetric sensitivity to various types of label noise, indicating that not all label noise impacts model performance equally. These insights provide a nuanced understanding of the limitations and capabilities of current WSAD approaches.

Limitations
The authors acknowledge that the benchmark may not cover all possible weak supervision scenarios and that the performance insights are based on specific datasets and experimental conditions. Additionally, the reliance on tabular foundation models may not generalize to all anomaly detection tasks, particularly those requiring domain-specific knowledge. The paper does not address the computational cost associated with the extensive experimentation, which may limit accessibility for some researchers.

Why it matters
The introduction of WSADBench is significant for the field of anomaly detection as it provides a structured approach to evaluate and compare various WSAD methods. By revealing the interdependencies among different weak supervision scenarios, this work encourages researchers to reconsider the isolation of these approaches and fosters a more integrated understanding of WSAD. The insights gained from this benchmark can inform the development of more robust algorithms and guide future research directions, ultimately enhancing the efficacy of anomaly detection in real-world applications.

Authors: Xu Yao, Siyuan Zhou, Wu Zhenbo, Chaochuan Hou, Shuang Liang, Shiping Wang, Hailiang Huang, Songqiao Han et al.
Source: arXiv:2605.26068
URL: https://arxiv.org/abs/2605.26068v1