OpenAI chief scientist urges slowdown as AI begins building smarter AI

AI major OpenAI’s chief scientist Jakub Pachocki has issued a striking call for voluntary industry slowdowns as artificial intelligence (AI) enters an era of recursive self-improvement where machines actively help build smarter machines.

The remark highlights a shift in the technological landscape, moving from human-coded software to systems that increasingly direct their own evolution. The urgency of this warning lies in a stark reality. If the speed of model capabilities continues to outstrip our ability to verify that these systems are safe, humanity risks losing control over a technology it cannot fully comprehend.

This concern represents a departure from the historical strategy of frontier laboratories. Around 2017, OpenAI internalised the consistent returns of scaling computational power, orienting its research to chase rapid capability growth. However, the creation of modern reasoning models has brought researchers face-to-face with the prospect of superhuman intelligence.

Recursive self-improvement occurs when an AI system begins to write its own software updates and improve its computational hardware. While this promises to accelerate scientific progress, it also creates an unpredictable feedback loop. 

Pachocki believes that the current path of unconstrained competition is unsustainable, noting that “the idea of racing forward at all costs seems absurd once one internalises the seriousness of the stakes”. To address this risk, he advocates for formal safety commitments, suggesting that development must be limited by strict safety thresholds and supervised by third-party auditors or international bodies.

A fundamental hurdle is that modern AI is not engineered in a traditional sense. Instead, it is grown experimentally through massive computational power, meaning its internal reasoning remains largely mysterious to its creators. 

In the post, Pachocki explains this phenomenon, saying “AI is grown more than designed - it is, to first degree, the product of repeating a straightforward optimization step many times on a hard-to-imagine amount of compute.”

This makes the task of alignment, which means ensuring that a system genuinely intends to act in accordance with human values, extremely difficult.

Researchers divide this safety challenge into two areas, namely goal alignment, which focuses on whether the system tries to perform its assigned task, and value alignment, which demands that the system acts with genuine integrity. 

Current alignment methods, such as training an AI with goal-oriented rewards, have proved brittle and prone to failure when systems face highly unfamiliar scenarios.

To track how reasoning models think, researchers have relied heavily on a process called chain-of-thought monitoring, which allows humans to read the step-by-step logic the model verbalises before answering. However, the post reveals that this critical window is actively shutting. As models integrate into complex multi-agent environments, learn to manipulate their own internal processes, and perform highly advanced tasks without needing to speak their thoughts aloud, the effectiveness of this monitoring is decaying. 

As a result, Pachocki warns, “No lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.”

This plea for caution comes during a broader industry transition toward highly autonomous agents. Frontier laboratories such as Anthropic, Google DeepMind, and Meta are actively competing to develop systems that can use tools, write complex software, and execute code independently. 

In response to these rapid capability jumps, the policy landscape is shifting. Several leading laboratories have voluntarily adopted responsible scaling policies that define specific danger thresholds. 

Meanwhile, governments are establishing AI safety institutes and debating international regulatory frameworks to enforce safety audits, echoing the growing consensus that unchecked development could pose severe systemic threats.