Jacob Coxon, a former researcher at Anthropic and OpenAI, has resigned from his position due to concerns that the rapid advancement of artificial intelligence could threaten human existence. Speaking with Fox News anchor Bret Baier, Coxon expressed a belief that there is a significant risk of an AI-driven catastrophe within the next decade.
A 10% chance of human extinction by 2030
Jacob Coxon, who spent three years conducting pretraining research at both OpenAI and Anthropic, believes the risk of AI causing human extinction is greater than 10% within the next ten years. As reported by Fox News, Coxon's departure stems from a belief that the industry is racing toward superintelligence without a viable plan for "alignment," the process of ensuring AI goals match human values.
While Coxon noted that the risks posed by current AI models are relatively low, he is deeply concerned about the trajectory of the technology. He argued that the current pace of development is so rapid that researchers are not being given the time required to solve the safety challenges inherent in superintelligent systems. This lack of caution, he suggests, is turning a solvable problem into a potential existential crisis.
Recursive self-improvement could arrive in as little as six months
The concept of "recursive self-improvement"—a process where AI systems begin to enhance their own intelligence—is a primary driver of Coxon's alarm... He suggested that this cycle of self-optimization could begin as early as next year, though he noted that some industry observers believe it could occur within the next six months. This rapid acceleration could lead to a sudden leap in capability that outpaces human control .
According to the report, the concern regarding this acceleration is shared across the industry. Even as companies like Anthropic attempt to address these risks, Coxon warned that the industry is not yet clearly on track to solve the alignment problem for superintelligence. The fear is that once an AI begins to improve itself, the window for human intervention may close permanently.
The danger of treating AI as an international 'superweapon'
Coxon described the current global competition for AI dominance as a dangerous arms race that treats the technology as an international "superweapon ." He argued that because the technology is being developed in a competitive, high-stakes environment, nations and corporations are prioritizing speed over safety. This creates a "hubristic gamble" where the winner of the race may inadvertently trigger a global catastrophe.
To mitigate this, Coxon emphasized that the global community must cooperate internationally to provide oversight. He noted that an uncoordinated arms race in artificial intelligence would be disastrous in a way that no other human activity has been in the past. Without a unified approach to regulation,the drive for technological supremacy may override the fundamental need for human survival.
The search for the 'somebody' who can regulate the race
While Coxon has called for international intervention,the specific identity of the regulatory body required to manage this transition remains unknown. He stated that "somebody" needs to step in to ensure the world does not race itself into extinction, but he did not name a specific organization or treaty that could fulfill this role. This leaves a massive gap between the identified need for oversight and the actual implementation of global rules.
Furthermore, the source indicates that while many technology leaders have publicly called for a "slow-down agreement," these pleas are frequently dismissed as mere marketing stunts. it remains unclear whether these industry leaders have the political capital to influence actual policy, or if the momentum of the AI race has already become too great to halt through traditional diplomatic or legal channels.
Comments 0