An OpenAI agent accessed non-public files from Australia’s Medicare, in what is believed to be the first reported incident of an AI agent hacking a government website.
Australian Prime Minister Anthony Albanese made the revelation during his appearance before the United Nations General Assembly in New York on Sept. 24. The data breach took place on June 18, became known to OpenAI in August, but was only communicated to the Australian government on Sept. 10 through a public mailbox.
“I spoke with the CEO of OpenAI, Sam Altman, to express Australia’s extreme concern about this incident. And I also expressed my disappointment that it took the company way too long to inform the Government what had occurred and the nature of the way that that notification occurred as well was unacceptable,” Albanese said during a press conference in New York.
Currently, there is no evidence that individuals were affected in the data breach, he said.
“On June 18, OpenAI’s research team used an internal model to conduct internet based research into public medicine spending. So, that’s how this began. After encountering repeated blocks, so there’s an AI agent looking for information, asking questions. There were blocks clearly which were coming back telling the AI agent, no. The AI agent found a way around those blocks. Didn’t accept no for an answer, if you like. The model attempted alternative ways to obtain the info that it wanted, and this led to unauthorised access into some other areas,” the PM explained.
Altman, who was also in New York, earlier told the UN Security Council about the dangers posed by AI, as he sought greater cooperation from governments, critical infrastructure operators, and technical experts.
“As AI systems become more capable and more autonomous, they could move faster than our institutions, concentrate power in too few hands, or make decisions that people no longer understand or control,” he said.
A task force has been formed in the wake of the data breach “to determine whether existing processes are appropriate to respond to AI-related cyber incidents,” Albanese said.
“The taskforce will be led by my department and involve the National Cybersecurity Coordinator, the Office of AI, the Australian Signals Directorate, the Australian AI Safety Institute, and Services Australia,” he noted.
According to a security expert, the gap isn’t code, but constraint, as frontier AI developers keep warning publicly about this specific risk while pushing each other toward more capable, autonomous systems.
“The latest OpenAI breach reflects a systemic failure across the industry and in governance. The U.S. has no federal AI law, the UK has no standalone AI legislation and the National Cyber Security Centre agentic AI guidance remains voluntary. The EU’s AI Act has just pushed its high-risk rules back to December 2027. Japan, Singapore and Australia all rely on voluntary guidelines rather than binding law. That leaves governments and organisations around the globe exposed to a threat moving at machine speed,” said Darren Guccione, CEO And Co-Founder, Keeper Security.
“Every AI agent is a new identity and a new attack surface. Enterprises must govern agents as they govern privileged users: with bounded scope, time-limited access, continuous monitoring and enforced policy. It requires treating agents with the same rigour organisations apply to human administrators. Without this governance, agent proliferation becomes uncontrollable privilege sprawl,” he added.
