OpenAI Medicare breach fuels push for tougher rules on rogue AI incidents

Companies could be forced to report AI hacks directly to cyber authorities under new national standards. ( Reuters: Dado Ruvic/Illustration/File Photo )

Australia is moving to ensure cyber authorities and affected organisations are alerted when AI agents cause security incidents.

Cyber experts warn reporting incidents will not be enough without stronger systems to detect and defend against them.

OpenAI's chief strategy officer Jason Kwon will travel from the US to front a parliamentary inquiry into artificial intelligence next week.

Tech companies would have to immediately report rogue AI incidents to both the affected organisation and Australia's cyber authorities under new standards being developed by the federal government.

OpenAI's months-long delay and low-level email contact alerting the government to a website breach has hardened Labor's resolve to impose a dual notification requirement for such cases.

But experts have warned mandatory reporting can only go so far, with calls for greater investment in cyber security to ensure Australia is able to detect and defend against artificial intelligence incursions.

The government launched a consultation paper to inform national AI standards earlier this month, which included the suggestion companies could be required to disclose certain incidents to "relevant Australian authorities".

The ABC understands the government now wants this to include notifying the Australian Signals Directorate (ASD) in addition to the relevant organisation subjected to the breach.

It took Services Australia five days to inform the ASD about a generic email from OpenAI informing the agency about an autonomous AI agent accessing non-public Medicare statistics from an old data portal.

The public inbox contacted by OpenAI was only monitored once a day, but Government Services Minister Katy Gallagher said the email address was now being monitored 24/7.

"We've strengthened that already," she said.

A rapid review into the OpenAI breach is due to conclude within "weeks", with the findings expected to inform the national standards legislation.

A joint parliamentary committee inquiry is also feeding into the laws, with OpenAI confirming its chief strategy officer, Jason Kwon, will fly from the US to appear at a hearing in Sydney next week.

Labor is hoping to introduce the legislation, which would also mandate standards for data centres, before the end of the year.

Chetan Arora, the director of education in software systems and cybersecurity at Monash University, said the OpenAI breach must be a "wake-up call" for Australia.

"While we hold them accountable and the onus is on these AI companies … we also need to build our own defence systems," he said.

Dr Arora said what is known as "zero-trust infrastructure" should be a "baseline requirement" for public-facing government systems.

"You design your systems so that you don't trust anybody by default."

He said it was "very clear" OpenAI had not taken sufficient steps to prevent its autonomous agents from repeatedly attempting to breach websites while undertaking tasks.

"Nobody can specify 100 per cent guardrails [or] think of every benign or malicious situation … but at least you try to cover the basics."

The rapid review into the OpenAI breach, under CEO Sam Altman, is expected to conclude within "weeks". ( Reuters: Brendan McDermid )

Dr Arora said Australia could mandate engineering standards, audit trails and other ways of tracking autonomous agents as a "condition of doing business" with the AI companies.

"We can regulate what kind of operations they run in Australia … [and] government itself is one of the largest AI customers generally in any country, so we can leverage that position."

He said "significant" investment in cyber security was also critical, including training the "next generation" of engineers and experts.

Treasurer Jim Chalmers said cyber security spending was an "ongoing feature" of budget considerations due to the "fast-moving" nature of the tech world.

"It's not like we waited for this event before we put a lot of time and effort and investment into safety in the AI world," he said.

The government is hoping to introduce legislation before the end of the year. ( ABC News: Joel Wilson )

Last budget the government allocated $160 million to improve the cyber security of Services Australia, with upgrades focused on protecting the most sensitive data first.

Opposition leader Angus Taylor on Monday said the government should be working "at pace" to get access to frontier AI models from the US.

"That's how we protect ourselves," he said.

"The best way to protect ourselves against cyber attacks is use those models for cyber defence."