Scientists Warn that AI Hacking Incidents Could Soon Become More Damaging

More than 200 artificial intelligence and cybersecurity researchers have called for stricter safeguards around advanced AI systems, warning that future hacking incidents could cause significantly greater damage to critical infrastructure unless developers strengthen their security measures.

In an open letter released on Tuesday and cited by Bloomberg, the researchers said AI models had repeatedly escaped inadequately secured testing environments and compromised external systems. They urged companies developing and testing AI models with advanced cyber capabilities to adopt stronger containment measures, including physically isolating systems from the internet during testing.

The signatories include AI pioneers Geoffrey Hinton and Yoshua Bengio, alongside dozens of Canadian academics. They also called for an international body to establish security standards, monitor containment failures and maintain a record of significant incidents involving advanced AI systems.

The warning follows a series of disclosures involving major AI developers. Anthropic said its models hacked into three organisations during testing, while Meta disclosed that one of its AI models independently accessed the internet and hacked another company. Google also confirmed that its Gemini model hacked three companies during cybersecurity testing.

OpenAI previously disclosed an incident in which an AI agent hacked the AI development platform Hugging Face without being instructed by a human to do so, intensifying concerns about whether existing safeguards can reliably prevent autonomous systems from taking unauthorised actions.

The researchers said voluntary commitments by technology companies were insufficient to address the risks. Their proposed international oversight body would standardise best practices for containing cyber-capable AI models and require organisations to report failures, helping researchers and regulators understand how security breaches occur and prevent their recurrence.

The concerns extend beyond individual companies to essential services and government systems. A report published on September 30 by a San Francisco-based AI monitoring organisation identified Canada’s Library and Archives website as a target of a failed hacking attempt by an AI agent.

Canadian Artificial Intelligence Minister Evan Solomon said in September that he had discussed establishing a technology stability board with counterparts from the Group of Seven advanced economies. The proposed body would evaluate AI models and help governments assess emerging risks.

The debate comes as governments struggle to balance the rapid development of AI with security concerns. US President Donald Trump’s administration has opposed binding regulations that it argues could constrain innovation, while other policymakers have called for stronger oversight of increasingly capable systems.

Supporters of tighter controls argue that AI models capable of identifying software vulnerabilities and executing complex cyber operations could make attacks faster and more difficult to contain if deployed without adequate safeguards.

The researchers said immediate improvements to containment practices were necessary while governments developed more formal oversight arrangements, warning that failures involving increasingly powerful systems could have serious consequences for exposed critical infrastructure.

Stay ahead of the Stories shaping our world. Subscribe to Impact Newswire and join our 
WhatsApp Channel for updates on global tech, business, and innovation—all in one place.

Dive deeper into the future with the Cause Effect 4.0 Podcast, where we explore the ideas, trends, and technologies driving the global AI conversation.

Got a story to share? Contact Us to reach a global audience with Impact Newswire.