AnchorReasoning: A Visual Grounding and Causal Reasoning Dataset in Long-Tail Autonomous Driving Scenarios

Problem

The paper addresses the gap in existing driving datasets that lack sufficient supervision for connecting visual evidence with reasoning and planning in autonomous driving scenarios. The authors highlight the need for a dataset that facilitates the understanding of decision-making processes in long-tail situations, where rare events may occur.

Method

The authors present the AnchorReasoning dataset, which is built on the WOD-E2E framework. It comprises 416,119 annotated frames and 395,379 decision-critical elements, categorized into four major categories and 19 fine-grained types. The dataset employs a Visually Grounded Chain-of-Thought (VG-CoT) structure that links element identification, attributes, implications, rationale, and planning. To enhance model performance, a curriculum supervised fine-tuning strategy is utilized, focusing on hierarchical capabilities. The evaluation metric introduced is an object-size-aware grounding metric, which assesses the localization quality of the identified elements.

Results

The results demonstrate significant improvements in various metrics: the 5-second Average Displacement Error (5-s ADE) decreased by 7.84, and the 5-second Final Displacement Error (5-s FDE) decreased by 11.86. Additionally, the RFS Frame metric showed an improvement of 1.66, while the RFS Cluster metric improved by 1.70. The model also achieved a reduction of 18.5 reasoning tokens on average, indicating a more efficient reasoning process. Furthermore, the inference latency was reduced by 0.32 seconds per frame on average. However, the paper does not specify the baselines against which these improvements were measured.

Limitations

The authors do not report any limitations in their work. However, the absence of specified baselines for the reported improvements may hinder the ability to fully assess the significance of the results.

Why it matters

The introduction of the AnchorReasoning dataset has implications for advancing research in visual grounding and causal reasoning within autonomous driving. By providing a structured approach to linking visual evidence with reasoning, it enables the development of more robust decision-making models that can handle rare and complex driving scenarios. This work lays the groundwork for future studies aimed at improving the interpretability and reliability of autonomous systems.