Anthropic Researcher Resigns After Warning AI Could Pose Deadly Risks Before 2030

Artificial intelligence researcher Jacob Coxon has left the AI industry after warning that leading companies including OpenAI and Anthropic are moving too quickly toward the development of self-improving AI systems. Coxon, 27, said both companies were failing to act responsibly in the race to develop increasingly capable models, arguing that the pursuit of artificial superintelligence could pose an extreme threat to humanity. Coxon spent the past three years working on the pretraining of AI models, first at OpenAI and later at Anthropic, which he had viewed as a company taking a more cautious approach to AI development. Pretraining is a foundational stage in the development of modern AI systems, during which models process enormous amounts of data to develop their ability to understand and generate information.

His decision to leave the sector comes as concerns about the potential risks of increasingly autonomous and capable AI systems continue to grow. In posts on X, Coxon accused both companies of “playing with our lives” by racing towards what he described as self-improving superintelligence. He said people involved in developing AI genuinely believe that advanced systems could potentially become capable of causing human extinction by the end of the decade. Coxon stressed that his warning was not intended as a publicity exercise, saying, “This is not a marketing stunt.” His comments reflect concerns among some AI researchers that systems capable of substantially improving their own capabilities could eventually become difficult for humans to control.

The concerns were reinforced by Anthropic safety executive Evan Hubinger, who responded publicly to Coxon's claims. Hubinger said Anthropic researchers do genuinely believe that AI could potentially “kill all humans” and estimated the probability of such an outcome at more than 10 per cent over the next decade. His assessment is a personal estimate rather than a prediction that such an event will occur, but it illustrates the seriousness with which some researchers within the AI safety community view the risks associated with highly advanced systems.

Also Read: Sanders Bill Could Jail OpenAI Anthropic CEOs For Developing Superintelligent AI Systems

At the centre of the debate is the concept of superintelligence, a theoretical stage at which an artificial intelligence system would possess capabilities that substantially exceed those of humans across a broad range of intellectual tasks. Researchers disagree sharply over how soon such systems could emerge, whether they would necessarily pose an existential threat and how effectively their behaviour could be controlled. Companies developing increasingly powerful models are simultaneously investing in safety research intended to identify and mitigate potential risks before systems become more capable.

Coxon's departure highlights a growing tension within the AI industry between accelerating technological development and addressing potential long-term dangers. His criticism is particularly notable because it comes from someone with direct experience working on advanced AI models at two of the industry's major companies. Hubinger's comments further underscore the extent of disagreement and concern within AI safety circles. As companies continue pursuing more capable and potentially autonomous systems, questions over how quickly development should proceed and what safeguards are necessary are likely to remain central to the debate surrounding the future of artificial intelligence.

Also Read: 22 BRS Legislators Face Suspension Following Disruption In Telangana Assembly