Parameter-Efficient Multi-View Proficiency Estimation: From Discriminative Classification to Generative Feedback

Problem
This paper addresses the gap in proficiency estimation from multi-view data, which is crucial for applications in coaching, rehabilitation, and talent identification. The authors highlight the challenges posed by the subtlety of proficiency indicators, such as timing and body mechanics, which are often distributed across multiple views and short temporal events. The work is presented as a preprint and has not yet undergone peer review.

Method
The authors propose three novel contributions to enhance multi-view proficiency estimation on the Ego-Exo4D dataset:

1. SkillFormer: A parameter-efficient discriminative architecture that employs selective multi-view fusion to optimize the integration of information from different perspectives while minimizing the number of trainable parameters.

2. PATS (Proficiency-Aware Temporal Sampling): This method improves temporal sampling by preserving locally dense excerpts of fundamental movements, ensuring that critical temporal features are retained for proficiency assessment.

3. ProfVLM (Proficiency Vision-Language Model): This reformulates proficiency estimation as a conditional language generation task, producing both a proficiency label and expert-style feedback. It utilizes a gated cross-view projector and a compact language backbone to facilitate this generative process.

The combined architecture achieves significant efficiency, with up to 20x fewer trainable parameters and up to 3x fewer training epochs compared to traditional video-transformer baselines.

Results
The proposed methods achieve state-of-the-art accuracy on the Ego-Exo4D benchmark. Specifically, the authors report that their approaches outperform existing baselines in proficiency estimation tasks, demonstrating a marked improvement in interpretability and feedback generation. The exact performance metrics are not disclosed in the abstract, but the authors emphasize the substantial reduction in computational resources required for training, which is a critical factor for real-world applications.

Limitations
The authors acknowledge that their methods, while efficient, may still be limited by the quality and diversity of the training data available in Ego-Exo4D. Additionally, the reliance on generative feedback may introduce variability in the quality of the feedback provided, depending on the model’s understanding of proficiency nuances. The paper does not address potential issues related to generalization across different action types or environments, which could affect the robustness of the proficiency estimates.

Why it matters
This work represents a significant advancement in the field of multi-view proficiency estimation, moving beyond traditional closed-set classification towards a more interpretable and actionable framework. The integration of generative feedback into proficiency estimation has implications for personalized coaching and rehabilitation strategies, enabling more tailored interventions based on individual performance metrics. Furthermore, the emphasis on parameter efficiency opens avenues for deploying these models in resource-constrained environments, such as mobile applications or real-time coaching systems.

Authors: Edoardo Bianchi, Antonio Liotta
Source: arXiv:2605.03848
URL: https://arxiv.org/abs/2605.03848v1