Study contradicts Anthropic and OpenAI claims that autonomous AI research is within reach

A recent study involving AI agents utilizing Claude Opus 4.8 and GPT-5.6 Sol aimed to assess the feasibility of autonomous AI research. Conducted by researchers from Princeton and the UK AI Security Institute, the experiment allocated six days, $3,000 in API credits, and GPU access for the AI to independently generate AI research papers. The results were evaluated by the original authors of unpublished NeurIPS papers, who rated the submissions as ‘Reject.’ This outcome highlights significant shortcomings in the AI’s performance.

The findings indicate that while frontier models can manage the technical aspects of the research engineering process, they struggle with critical elements such as research judgment, creative problem-solving, and the capacity to abandon unproductive approaches. These limitations suggest that the current state of AI technology is not yet capable of achieving the level of autonomy in research that organizations like Anthropic and OpenAI have suggested is within reach. The study underscores the gap between technical capabilities and the nuanced understanding required for effective research.

This research challenges the optimistic narratives surrounding the potential for autonomous AI in research settings, emphasizing the need for further advancements in AI’s cognitive and evaluative capabilities. For more details, refer to the original article on The Decoder.