The race to build increasingly powerful artificial intelligence systems has taken an unusual turn, with some of the biggest names in the industry agreeing that the pace of AI development may need to slow down. Anthropic CEO Dario Amodei has called for companies to “pace the frontier” of AI development, receiving public support from OpenAI CEO Sam Altman and Elon Musk, who leads xAI.
The unusual agreement comes as AI models become increasingly capable of operating autonomously, carrying out complex tasks and interacting with external systems. At the same time, recent incidents involving AI models being used for hacking, surveillance and fraud have intensified concerns about whether safety measures are keeping up with technological progress.
Why AI Leaders Want a Slower Pace
Amodei is not calling for an end to AI development or a complete halt to model training. Instead, his argument is that AI companies should slow the rate at which capabilities are advanced when safety systems cannot keep pace.
In an essay published over the weekend, Amodei proposed a three-part framework. The first step would involve independent safety evaluators being given permanent, employee-level access to AI companies so they can assess whether safety commitments are actually being followed.
The second would involve leading AI companies coordinating on common safety standards and limits on unchecked development. The third would require international cooperation to manage the risks posed by increasingly powerful AI systems.
Amodei’s concerns are partly based on the increasing autonomy of AI agents. He warned that within six to 12 months, coordinated AI agents could potentially become capable of causing enormous damage online if their capabilities continue advancing without sufficient safeguards.
OpenAI and xAI Back the Idea
The proposal is notable because Anthropic, OpenAI and xAI are competitors in the rapidly expanding frontier-AI market.
OpenAI CEO Sam Altman publicly supported Amodei’s proposal, saying OpenAI would also commit to independent evaluators having significant access to assess its safety practices. Elon Musk, whose xAI is developing its own advanced AI models, also expressed agreement with the need for greater caution. Alphabet’s Google DeepMind chief Demis Hassabis has also offered support for the broader safety discussion.
Such agreement is unusual because these companies have strong financial and competitive incentives to continue improving their models. The AI industry is attracting hundreds of billions of dollars in investment, while companies are racing to develop systems that can perform increasingly sophisticated reasoning, coding and autonomous tasks.
That creates a difficult balance: slowing down may improve safety, but moving too slowly could allow competitors or rival countries to take the lead.
Recent AI Incidents Have Increased Concern
The calls for caution come after several incidents that have raised questions about how autonomous AI systems could behave outside controlled environments.
Anthropic recently reported cases in which its Claude models were used for activities involving cyber operations, surveillance and fraud. Separately, concerns grew following an incident involving OpenAI agents and the open-source platform Hugging Face, in which a group of AI agents reportedly carried out unauthorised cyber activity.
These incidents do not mean that AI systems are already uncontrollable. However, they demonstrate how quickly AI agents can move from simply generating information to taking actions using digital tools.
That distinction is becoming increasingly important. A chatbot that answers a question presents one type of risk, while an autonomous system capable of writing code, accessing online services, coordinating with other agents and executing tasks presents a much broader one.
The AI Industry Faces a Difficult Choice
The biggest challenge is that there is no universally accepted definition of how much AI development should slow down.
Companies still want to improve models because more capable systems could deliver major benefits in areas such as scientific research, medicine, education, software development and productivity. At the same time, developers need enough time to test these systems and understand their behaviour before releasing increasingly autonomous capabilities.
There is also a geopolitical dimension. Amodei has argued that any slowdown among democratic countries should be designed carefully so that the United States and its allies do not lose their technological advantage to countries such as China. He has also called for stronger controls around advanced AI chips and techniques that could allow competitors to rapidly reproduce the capabilities of more advanced models.
This makes a global AI agreement particularly difficult. Governments may agree that AI safety matters, but they may not agree on how much development should be restricted or who should enforce the rules.
What the Slowdown Debate Means for AI Stocks
The debate has already affected financial markets. AI-linked stocks and semiconductor companies came under selling pressure on September 14 after the weekend warnings. Nasdaq-100 futures fell sharply, while Nvidia declined more than 2% in pre-market trading. Asian AI-related companies, including Kioxia, SK Hynix and Samsung Electronics, also faced significant losses.
The concern for investors is not necessarily that AI is disappearing. Instead, markets are worried that even a modest delay in AI development could affect the enormous spending plans built around data centres, advanced chips and computing infrastructure.
Reuters Breakingviews noted that a shift from training increasingly powerful models toward using existing models at scale could actually create opportunities for AI inference and applications. McKinsey estimates that inference could account for a much larger share of data-centre demand by 2030.
Therefore, the latest debate may represent a change in the direction of AI investment rather than the end of the AI boom.
The coming months will show whether the industry’s call for caution results in genuine changes to development timelines or simply leads to stronger safety testing alongside continued technological progress. For investors, the key question will be whether companies can maintain AI’s enormous economic potential while proving that increasingly powerful systems can be developed and deployed safely.
About The Author
