Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Problem

This work addresses the gap in reliable robot capabilities across diverse tasks, which traditionally require significant human effort for skill development, reward design, and the integration of perception and control. The authors highlight the need for a systematic approach to enable robots to autonomously improve their skills in a variety of manipulation tasks. The paper is a preprint and has not yet undergone peer review.

Method

The proposed framework, Reconstruct, Practice, Go Real (RPG), leverages an offline dataset of manipulation capabilities to construct related practice tasks. The method incorporates a feedback mechanism that utilizes execution feedback, privileged simulator state, and videos from the dataset to diagnose failures encountered during task execution. RPG focuses on developing new reusable symbolic skills, refining existing skills, and revising the system prompt to enhance performance. The evaluation method involves cross-task evaluation, testing individual candidate changes and merged revisions to assess their impact on task success. A multimodal large language model (LLM) is employed to coordinate perception and robot control, utilizing the system prompt and a skill library to facilitate this integration.

Results

The RPG framework demonstrates a task success rate of 95.0% after 15 practice rounds, a significant improvement from 28.6% after the first round. In benchmark comparisons, RPG outperforms ASPIRE, which achieved a success rate of 75.5%, and CaP-Agent0 powered by GPT-6 Astra Pro, which reached 60.0%. Additionally, the framework succeeded in all 30 physical trials conducted across three distinct tasks, showcasing its robustness in real-world applications.

Limitations

The authors do not report any limitations in the study. However, as with any preprint, the absence of peer review may mean that potential weaknesses or areas for improvement have not been fully explored or acknowledged.

Why it matters

The implications of this work are significant for the field of robotics, particularly in the development of autonomous systems capable of self-improvement. By providing a structured approach to skill development and integration of perception and control, the RPG framework could pave the way for more versatile and capable embodied agents. This could enhance the deployment of robots in complex environments, reducing the reliance on human intervention for skill acquisition and task execution.