FTC launches probe into OpenAI, Anthropic over ‘rogue’ AI agents as concerns grow over safety

The US Federal Trade Commission (FTC) has launched an industry-wide investigation into leading artificial intelligence companies including OpenAI and Anthropic, amid growing concerns about the potential risks posed by increasingly autonomous AI agents.A senior FTC official told Reuters on Wednesday (September 30) that the agency plans to issue formal demands for information and compel testimony from executives at major AI developers, including OpenAI, Anthropic and AI research group METR.

The investigation is examining the potential dangers the companies' technology could pose to consumers.The probe comes after a series of incidents involving AI agents carrying out unexpected or unauthorised actions, including cybersecurity-related activity. It marks a major US enforcement effort focused on the safety of so-called “rogue” or autonomous AI agents.FTC turns focus to AI safetyFTC Chairman Andrew Ferguson had concerns about AI companies before an OpenAI agent-related incident involving the open-source AI platform Hugging Face, according to the senior official.The incident increased the urgency around the issue, as OpenAI agents reportedly probed Hugging Face for vulnerabilities before carrying out a large-scale attack.

The FTC is now seeking information from companies and researchers involved in developing and evaluating agentic AI systems.The agency has broad authority under US consumer-protection law to pursue companies over unfair or deceptive practices. It has previously used those powers in cases involving inadequate measures to protect consumer data.The investigation also comes as policymakers debate who should bear responsibility when autonomous AI systems cause harm or act beyond their instructions.What did the “rogue” AI agents from OpenAI, Anthropic and Meta do?

A series of incidents involving AI agents from OpenAI, Anthropic and Meta has raised concerns about whether increasingly autonomous systems can remain within the limits set by their developers.OpenAIOpenAI’s agents were involved in several incidents:Hugging Face: During cybersecurity testing in July, OpenAI models bypassed controls designed to isolate them from the internet, gained access to third-party systems and compromised parts of Hugging Face’s infrastructure.

OpenAI described the incident as its most serious model-related hack identified at the time.Australian government portal: An OpenAI agent accessed non-public files on an Australian government health-statistics portal while attempting to retrieve information. OpenAI said no personal health data was accessed.US government websites: OpenAI agents searching federal websites behaved in ways that went beyond their instructions.

In one case, they found API developer keys, although only publicly available information was ultimately gathered. OpenAI has also been reviewing other incidents involving government websites.AnthropicAnthropic said its AI models hacked into three organisations during controlled cybersecurity testing. The company discovered the incidents after reviewing more than 141,000 evaluation runs as part of a large-scale security review.The models involved included Claude Opus 4.7, Claude Mythos 5 and an internal research model.

Anthropic said the earliest incidents dated to April and involved models accessing the internet from testing environments that were supposed to be isolated.MetaMeta disclosed that one of its AI models accessed the internet and hacked another company during cybersecurity testing.Meta said the incident resulted from a misconfiguration by Irregular, the independent company conducting the testing, which inadvertently allowed the model to access the internet.

The episode nevertheless added to concerns about AI systems taking actions beyond their intended testing boundaries.Who is liable when AI goes rogue? The incidents have intensified debate over responsibility when an AI agent performs an unauthorised action.Ferguson has rejected the idea of treating AI agents as independent actors that simply “break loose”, arguing that responsibility can instead lie with the developers or users who instruct the systems.That question could become increasingly important as AI companies deploy agents capable of operating autonomously for extended periods and accessing external systems.At the same time, President Donald Trump has pushed for US leadership in AI and has argued against excessive regulation, while also saying existing laws can be used to hold AI companies responsible for harm.Trump met AI executives this week, with leading companies agreeing to voluntary standards for AI development.

The FTC probe adds a formal regulatory dimension to those industry-led efforts.The investigation is therefore unfolding as AI developers simultaneously expand the capabilities of autonomous agents and face growing pressure to demonstrate that those systems can be safely controlled.( With Reuters, AP inputs)