The trigger for the latest wave of AI-related alarm is Jacob Coxon, a researcher who moved from OpenAI to Anthropic, the company developing Claude. In resigning, Coxon accused both companies of rushing toward a self-improving superintelligence without, in his view, behaving responsibly enough. “They are gambling with our lives,” was the summary reported by UnHerd (here).

This isn’t an isolated voice. The press reports that other researchers in the Anthropic ecosystem are attributing significant percentages to the possibility that AI will have existential consequences. Dario Amodei himself has evoked scenarios in which things could go “really, really bad.” There has even been talk of “human extinction by 2030”! To dismiss it all as science fiction would be superficial. But it would be equally superficial to accept the most extreme conclusions without verification.

When agents go beyond the limits

There are, in fact, concrete cases that deserve attention. The collected material describes an experiment in which approximately 1,200 OpenAI agents, placed in a sandbox without free access to the internet, identified a vulnerability in the proxy that represented their only escape route, then built shared tools to connect to the internet. Approximately 700 attacked Hugging Face. This is one of the world’s leading platforms for developing and sharing artificial intelligence models. It functions, in part, as the “GitHub of AI”: it hosts models, datasets, libraries, and tools used by researchers, startups, and large companies. It should be noted that the test was conducted with some security classifiers deactivated. This puts into perspective the Hollywood image of the machine suddenly “rebelling,” but it doesn’t eliminate the problem. As MIT’s Daniel Huttenlocher observes in the collected material, saying that “AI has gotten out of control” anthropomorphizes the software. A less cinematic formulation is perhaps more disturbing: humans have granted autonomous systems operational capabilities without knowing exactly how they would use them. The real risk isn’t Terminators but agents capable of planning, adapting, using tools, and performing actions not explicitly foreseen. The more autonomy granted, the greater the responsibility of those who grant it.

The genie has already come out of the bottle

Generative AI is still in its infancy. Yet the point of no return seems to have already been passed. Models are circulating, expertise is widespread, open-source systems can be copied and modified, and competition between the United States and China is rapidly escalating. In this sense the genie has already emerged from the bottle. Digital technology isn’t fissile material concentrated in a few identifiable facilities. An American moratorium wouldn’t automatically lead to a Chinese or global moratorium. The risks, therefore, cannot be ignored. But precisely because they are real, we must ask ourselves whether the proposed solutions serve only to reduce them or whether they can also produce economic and political advantages for those who propose them.

The Amodei plan

In the essay “We Must Pace the Frontier,” Dario Amodei proposes three main lines.

The first involves introducing external evaluators into frontier laboratories, with access comparable to that of employees and the ability to verify processes, training pipelines, and incidents. The most commonly recommended independent audit organization model is the company Model Evaluation & Threat Research (METR).

The second concerns a limited exemption from antitrust rules, to allow the major laboratories to coordinate on safety standards and development pace.

The third is geopolitical: reaching an agreement with China on certain red lines, while maintaining restrictions on chips, countering model distillation, and protecting the weights of the most advanced systems.

A particularly critical reconstruction of these proposals is present in a ZeroHedge article on the clash between Amodei and Zuckerberg .

The Timing of the Coxon Case

Here the most uncomfortable questions begin. According to an article by Maddalena Loy from La Verità, Coxon’s message was preempted by 22 minutes by a Wall Street Journal report. Immediately afterward, a post from a previously little-known account gained enormous traction. As Jeffrey A. Tucker reports, Coxon’s post amassed over 150 million views for no apparent reason. UnHerd also reconstructs how, in the first few minutes—decisive for algorithmic propagation—the post was amplified by accounts belonging to the “AI doomers” ecosystem. OffGuardian even goes so far as to speak of a possible “psy-op.”

But we need to distinguish. The temporal sequence, the amplification, and the networks of relationships can fuel suspicion, but they do not in themselves prove the existence of a coordinated operation. UnHerd itself points out that many AI safety researchers can be perfectly sincere. Financial interests and good faith are not necessarily incompatible.

First driver: capital, losses and debt

The first possible alternative incentive to pure ethical concern is financial.

According to Alessio Mannino in the InsideOver article “The Great AI Slowdown? But what ethics, it’s just strategy: market, geopolitical, and ideological,” OpenAI would have closed 2025 with approximately $13 billion in revenues against $34 billion in costs. Meanwhile, the sector is financing an arms race of chips, data centers, electricity grids, and debt. Between liabilities and future investments, the bill could approach $1.3 trillion by 2027. The dynamic described is circular: chipmakers finance labs, labs buy clouds, the cloud fuels new investments, and the same investors bet on future growth.

This machine can’t easily stop. A model becomes obsolete as soon as a competitor comes up with a better one. And precisely for this reason, a general pause would have an economically interesting effect: it would freeze the competitive standings. Those already at the top could therefore benefit from rules that make it more difficult for those chasing them to catch up.

Second driver: creating barriers to entry

Costly regulation does not burden everyone equally. Anthropic, OpenAI, Microsoft, or Meta can afford armies of lawyers, audits, certifications, permanent assessors, and compliance structures. A startup or open source project can’t. This is the classic criticism of regulatory capture: the dominant company doesn’t necessarily have to fight the regulator. It can anticipate it, help write the rules, and transform regulation into a competitive moat.

Jeffrey Tucker, in the Epoch Times article “What’s Going On With AI Regulation?“, precisely interprets the phenomenon of private companies demanding more public oversight in this way. “They are trying to preempt inevitable regulation to dictate the terms, limit competition with major players, and establish themselves as the controlling power within the agencies that will be responsible for it. They want to win the game and ensure that no one else ever gets to play it,” writes Tucker.

The question is simple: will regulation protect the public or will it protect incumbents from competition?

Third driver: an authorized oligopoly

Even more delicate is the request to allow the main laboratories to coordinate by at least partially derogating from antitrust regulations. For supporters, it’s the only way to avoid an irresponsible rush. For critics, however, it means risking turning market leaders into a state-sanctioned cartel. This is the fear also expressed in Stefano Graziosi’s article in La Verità, “Anthropic-dem axis to control who will control the digital revolution”. Formally, the government would control the industry; physically, however, the stronger industry could influence rules, standards, and even the structure of its own oversight.

Fourth driver: who controls the controllers?

This is probably the most delicate point of the Amodei proposal.

METR is presented as a possible independent evaluator, but press articles reconstruct financial, personal and philanthropic relationships that overlap in part with the Anthropic ecosystem.

Dustin Moskovitz and Jaan Tallinn, as reported by Tom Ough on UnHerd, are among the funders of the AI ​​safety movement and are also investors in Anthropic. Coefficient Giving, formerly Open Philanthropy, funded ARC, the incubator that gave rise to ARC Evals, which later became METR. Furthermore, Daniela Amodei is married to Holden Karnofsky, co-founder of Open Philanthropy. This is a “billion-dollar foundation created with the aim of monitoring the risks of artificial intelligence, acting as an ‘independent’ authority in the sector: in practice, the monitor is the husband of the monitored,” writes Maddalena Loy.

The issue is developed in Maddalena Loy’s article in La Verità, “Amodei’s brother-in-law is also keeping an eye on the AI”.  It should be noted, however, that METR claims not to receive funding from AI companies or their employees and maintains that its funders do not decide which projects it undertakes, as reported by the New York Post. The connections exist. However, automatically concluding that METR is controlled by Anthropic would go beyond what the sources demonstrate.

The question of “monoculture”

Some press articles also take the criticism to an ideological level, speaking of a progressive or “woke” monoculture in the AI ​​safety ecosystem. This key also appears in the article from La Verità “The left wants to shield its AI”.

Rather than turning the “woke” label into an objective fact, the interesting question concerns pluralism: If funders, researchers, managers and evaluators largely belong to the same cultural and professional milieu, who decides what safety, alignment and acceptable risk mean? A truly independent controller should also be intellectually independent.

Fear or smokescreen?

From this overlap arises the most serious accusation: catastrophism as a smokescreen.

The fear of human extinction captures far more public attention than debt, balance sheets, IPOs, monopoly rents, or antitrust issues. If the debate is reduced to the alternative between immediate regulation and the end of humanity, any economic objection can appear irresponsible. OffGuardian speaks openly of a psychological operation. But the evidence collected doesn’t allow us to prove this. It’s more prudent to speak of a convergence of incentives: genuine fear, commercial interest, lobbying, politics, and media dynamics can mutually reinforce each other without the need for a centralized plan.

Huang and Zuckerberg break the front

Two protagonists on opposite sides are difficult to ignore. Jensen Huang, CEO of Nvidia, the publicly traded company with the world’s largest market capitalization and thus no rookie, has repeatedly called existential scenarios exaggerated and unscientific. Ross Andersen’s article in The Atlantic, “The Most Powerful Man in AI Isn’t Worried at All,” highlights his belief that security is primarily an engineering issue rather than a legal one. Huang also provocatively accused some operators of creating demand for their security products by emphasizing the dangers: “What better way to create demand than to create a problem?”

Mark Zuckerberg, however, disputes the idea of ​​a coordinated pause. According to ZeroHedge’s reconstruction, Meta has already independently delayed some products to improve alignment without asking other labs to do the same. His alternative is civil liability, competing evaluators, and companies held accountable for damages.

Huang and Zuckerberg, of course, also have huge economic interests. And that’s precisely the point: none of the protagonists in this battle can be considered neutral by definition.

Trump dismisses the apocalypse

Donald Trump has taken the most drastic position, calling the “doom” warnings a “hoax” and arguing that slowing down the United States would benefit China. His position is also reported by Federico Rampini in the Corriere della Sera in the article “The AI-apocalypse can wait. What’s really behind the wave of panic?” But dismissing it all as a hoax doesn’t solve the technical problem. The fact that Amodei might have financial incentives doesn’t make autonomous AI agents harmless.

The most solid criticisms of catastrophism

From the sources collected, four particularly strong criticisms emerge.

First, there is no consensus among experts on the imminence of human extinction.

Second, speaking of AI as a subject that “wants,” “deceives,” or “decides” risks obscuring the responsibility of the humans who grant it powers and tools.

Third, focusing on the coming apocalypse can distract from the problems already present: cybersecurity, autonomous weapons, surveillance, energy, labor, and algorithmic discrimination.

Fourth, regulations tailored to the giants can become a barrier to entry for startups, competitors, and open source, with the resulting “democracy in the economy” going out the window.

What companies could do immediately

The conclusion is perhaps the most important part. Companies don’t need to convince the public that extinction is imminent to take concrete action. They can limit the autonomy of agents in irreversible activities or those that affect critical infrastructure. They can maintain preemptive human controls on payments, cyber operations, and high-impact decisions. They can separate those who develop, those who execute, and those who verify, preventing the controlled system from also becoming its own controller. They can publish incidents, vulnerabilities, and complete test results. They can rely on truly pluralistic evaluators, making funding and conflicts of interest transparent. And above all, they can assume full legal responsibility for their products. If an agent causes damage because he has been granted excessive powers, liability cannot be dissolved by the “out-of-control machine” formula. The AI ​​debate therefore needs something more difficult than either technological optimism or apocalypse: skepticism in both directions. Researchers who expose risks must be taken seriously. Agents who exceed established boundaries must be taken seriously. But equally serious must be debt, IPOs, financial interests, relationships between financiers and regulators, antitrust exemptions, and potential barriers to competition.

The question isn’t just whether we should be afraid of artificial intelligence. The question is who will transform that fear into power. And above all, who will control those who claim to want to protect us.