Learning-Guided Planning in Large Dynamic Action Spaces: Budgeted Tree Search for One-to-Many Mobile Charging

Problem

The paper addresses the challenge of one-to-many mobile charging in environments with numerous candidate actions, specifically in large dynamic action spaces. The authors highlight the limitations of existing methods in efficiently selecting charging stops from a vast set of options, which can range from 736 to 2,813 stops. This work is presented as a preprint and has not undergone peer review.

Method

The authors propose the Learning-Guided Planning architecture (LP-BTS), which integrates several components to enhance decision-making in mobile charging scenarios:

• Graph Proposal Policy: This component focuses on a small candidate support set to streamline action selection.
• Value Critic: It evaluates the leaves of the decision tree to inform the selection process.
• Edge-Budgeted PUCT: This method allows for the comparison of short simulated futures before committing to an action, optimizing the decision-making process.

The training data consists of 250 sensors and 1,125 candidate charging-stop actions, enabling the model to learn effective strategies for action selection. The architecture is evaluated through ablation studies comparing uniform sampling against targeted support, revealing insights into the effectiveness of the proposed methods.

Results

The proposed LP-BTS architecture achieves notable performance metrics:

• Survival Rate: 0.4545, the highest observed in the study.
• Alive-AUC: 0.8031, indicating a strong performance in maintaining operational sensors.
• Survival Advantage: +0.0066 (95% CI [-0.0037, +0.0184]) when compared to the strongest domain-engineered comparator, although the authors note that the difference in survival advantage remains unresolved.
• Distance Traveled: The model enables a 23% increase in distance traveled through direct policy selection.
• Survival Percentage Points Lost: The use of uniform sampling results in an 8.8 percentage point loss, while targeted support fixed this loss to 1.4 percentage points.

Limitations

The authors acknowledge an unresolved difference in survival advantage over the strongest domain-engineered comparator, which may indicate a need for further investigation into the robustness of their approach. Additionally, the reliance on a specific training dataset may limit generalizability to other contexts or environments.

Why it matters

This work has significant implications for the field of mobile charging and dynamic action planning, particularly in scenarios with large action spaces. The proposed LP-BTS architecture not only demonstrates improved performance metrics but also provides a framework for future research in optimizing decision-making processes in complex environments. The findings could influence the design of more efficient algorithms for similar applications, potentially leading to advancements in autonomous systems and resource management.