DataMaster: Towards Autonomous Data Engineering for Machine Learning

Problem
This paper addresses the gap in autonomous data engineering for machine learning, a process that remains predominantly manual and ad hoc. As model architectures and training methodologies become standardized, the bottleneck shifts to data quality and selection. The authors propose a framework, DataMaster, to automate data engineering tasks such as external data discovery, selection, cleaning, and transformation, thereby enhancing downstream model performance without altering the underlying learning algorithm. This work is presented as a preprint and has not yet undergone peer review.

Method
DataMaster is a data-agent framework designed to optimize data engineering through a structured approach. It comprises three main components:

1. DataTree: A tree-structured search mechanism that organizes various data-engineering branches, allowing for systematic exploration of data options.
2. Shared Data Pool: A repository that stores discovered external datasets, facilitating reuse across different experiments and reducing redundancy in data sourcing.
3. Global Memory: A cumulative memory system that records outcomes, artifacts, and findings from previous branches, enabling the agent to leverage past experiences in future data selections.

The framework employs a task-conditioned approach, where the agent iteratively refines its data choices based on feedback from downstream training, thus optimizing the data side of the learning process. The authors do not disclose specific details regarding the training compute or the exact architecture of the agent.

Results
DataMaster was evaluated on two benchmark suites: MLE-Bench Lite and PostTrainBench. On MLE-Bench Lite, DataMaster achieved a 32.27% improvement in medal rate compared to the initial score, indicating a significant enhancement in data-driven model performance. In the PostTrainBench evaluation, DataMaster outperformed the instruct model on the GPQA task, achieving a score of 31.02% versus 30.35%. These results demonstrate the effectiveness of the proposed framework in optimizing data for machine learning tasks.

Limitations
The authors acknowledge several limitations, including the potential for overfitting to specific datasets due to the reliance on historical data stored in the Global Memory. Additionally, the framework’s performance may vary depending on the diversity and quality of the external datasets available in the Shared Data Pool. The paper does not address the computational overhead introduced by the tree-structured search and memory management, which could impact scalability in larger applications.

Why it matters
The implications of this work are significant for the field of machine learning, particularly as data quality becomes increasingly critical for model performance. By automating data engineering, DataMaster could streamline the workflow for practitioners, allowing them to focus on model development rather than data curation. This framework could serve as a foundation for future research into autonomous systems that enhance data-driven decision-making in machine learning, potentially leading to more robust and efficient models.

Authors: Yaxin Du, Xiyuan Yang, Zhifan Zhou, Wanxu Liu, Zixing Lei, Zimeng Chen, Fenyi Liu, Haotian Wu et al.
Source: arXiv:2605.10906
URL: https://arxiv.org/abs/2605.10906v1