Game-Guided Skill Discovery through Self-Play for Playable Agent Control

Problem

This work addresses a gap in existing unsupervised skill-discovery methods, specifically focusing on the semantic distinctness, interpretability, and expressivity of motor skills. The authors highlight that current approaches lack the ability to generate skills that are both human-interpretable and effective in diverse tasks. This paper is a preprint and has not undergone peer review.

Method

The authors propose a framework called Game-Guided Skill Discovery (GGSD), which utilizes self-play in competitive gameplay to facilitate skill discovery. The architecture consists of a hierarchical agent structure that includes:

• High-level Policy: This component selects from a small discrete set of skills, allowing for a structured approach to skill selection.
• Low-level Policy: This skill-conditioned policy learns the corresponding behaviors associated with the selected high-level skills.

The training environments utilized in this study include the Ant, Franka-arm, and Unitree G1, providing a diverse set of scenarios for skill application. Notably, the high-level policy can be replaced by a human operator, enabling direct control and interaction with the agent.

Results

The framework demonstrates a significant capability in skill composition, where the learned human-playable skills can effectively solve previously unseen tasks, such as Maze and CubePush, without requiring additional training. The paper does not report any quantitative results or comparisons against established baselines.

Limitations

The authors do not report any limitations in their work. However, the absence of quantitative results and comparisons to existing methods may limit the assessment of the framework's performance relative to other skill-discovery approaches.

Why it matters

The implications of this work are substantial for downstream applications in robotics and AI, particularly in enhancing the interpretability and effectiveness of learned motor skills. By enabling agents to discover and utilize skills that are both human-playable and adaptable to new tasks, this research paves the way for more intuitive human-agent interactions and broader applicability in real-world scenarios.