Why recursive AI remains a distant goal for the industry

The concept of recursive self-improvement (RSI)—where an AI system continuously upgrades itself—has become the latest focal point for AI development. While the term has become a three-letter byword for a cataclysmic AI takeoff, the actual technology remains in its infancy. Earlier this month, AI researcher Richard Socher launched a startup named Recursive Superintelligence to pursue this explicit goal. Socher stated, “Our main focus is to build truly recursive, self-improving superintelligence at scale, which means that the entire process of ideation, implementation, and validation of research ideas would be automatic.”

Several prominent researchers and startups are pursuing similar automation goals, though their current progress relies heavily on human-directed engineering. AI researcher Andrej Karpathy has been using agent swarms to train large language models on simple tasks for a project called Auto-Research. However, in March, Karpathy noted that the project was not yet novel or ground-breaking research. Similarly, the startup Adaption, founded by Sara Hooker, launched a tool called AutoScientist to automate frontier training. Meanwhile, Doris Xin, founder of the startup Disarray, saw her machine learning agent win 28 medals in a Kaggle competition. Xin argued that given infinite compute and an infinite time horizon, the industry is already there, describing the work as a matter of engineering rather than a creative endeavor.

Despite these developments, the AI industry is not very close to recursive systems in any meaningful way. Google CEO Sundar Pichai noted that while progress is being made, the kind of acceleration described as RSI is not quite here yet. Current tools still require human oversight. In January, an Anthropic lead programmer estimated close to 100% of his team’s code was written by Claude Code. A survey regarding Anthropic’s Mythos preview showed that five out of 18 Anthropic engineers believed that, with harness improvements, this version of Mythos could soon substitute for an L4 engineer—a midlevel programmer who can work without supervision. However, the model still struggled with self-direction, which is the cornerstone of true RSI.

Experts from the Center for Security and Emerging Technology (CSET) express skepticism. Helen Toner, director of CSET, pointed out that simply using AI tools to do AI research is different from the classic definition of RSI, which requires that no humans are needed. Ajeya Cotra, a researcher at METR, an AI research organization, distinguished between “adequacy” (performing research without humans, even if less efficiently), “parity” (matching human capability), and “supremacy” (outperforming human-AI collaboration). While AI may be close to adequacy, achieving true parity remains a longer-term challenge.

Why it matters

The industry is pivoting toward recursive self-improvement as a North Star, but the gap between current human-directed AI tools and fully autonomous systems remains significant.