AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning

Problem
This paper addresses the limitations of Class-Incremental Learning (CIL) in the context of CLIP-based models, specifically focusing on the challenges of attribute extraction and aggregation when learning new classes. The authors highlight that existing methods often lead to catastrophic forgetting due to the inability to effectively manage the shared representation space as new classes are introduced. This work is presented as a preprint and has not yet undergone peer review.

Method
The proposed method, AREA (Attribute Extraction and Aggregation), introduces a two-stage approach to enhance CLIP-based CIL. For attribute extraction, the authors utilize principal geodesic analysis to anchor class-level visual and textual attributes within a hyperspherical embedding space, which stabilizes the extraction process. For attribute aggregation, AREA employs lightweight task-specific experts that utilize scoring and residual refinement, regulated by a variational information bottleneck objective to mitigate overfitting and maintain generalization across tasks. During inference, the model routes over task attribute manifolds using optimal transport, allowing for more concise and accurate predictions. The architecture leverages existing CLIP embeddings, but the specifics of the training compute and dataset used are not disclosed.

Results
The authors report that AREA consistently outperforms state-of-the-art (SOTA) methods on benchmark datasets for CIL. While specific numerical results are not provided in the abstract, the paper claims significant improvements in performance metrics compared to existing approaches, indicating a robust effect size in mitigating catastrophic forgetting and enhancing classification accuracy across incremental tasks.

Limitations
The authors acknowledge that their method may still be susceptible to certain forms of forgetting, particularly in scenarios with highly overlapping class attributes. They also note that the reliance on optimal transport for routing may introduce computational overhead, which could limit scalability in real-time applications. Additionally, the paper does not discuss the potential impact of varying dataset sizes or the effects of domain shifts on the performance of AREA.

Why it matters
The implications of this work are significant for advancing CIL methodologies, particularly in applications where continuous learning from diverse and evolving data streams is critical. By effectively managing attribute extraction and aggregation, AREA provides a framework that could enhance the robustness of CLIP-based models in real-world scenarios, such as autonomous systems and personalized AI applications. This research opens avenues for further exploration into hybrid models that can leverage both visual and textual information more effectively, potentially leading to more sophisticated and adaptable AI systems.

Authors: Zhen-Hao Xie, Yu-Cheng Shi, Da-Wei Zhou
Source: arXiv:2605.28809
URL: https://arxiv.org/abs/2605.28809v1