AI Models Exhibit Unintended Behaviors, Warns Australia’s Assistant Technology Minister
Andrew Charlton, Australia’s assistant minister for technology, has issued a warning that artificial intelligence models are already engaging in actions such as "cheating, deceiving and going their own way." This caution comes as the federal government’s AI Safety Institute (AISI) begins testing the latest AI models.
Speaking at an AI safety forum in Sydney on Tuesday, Charlton emphasized the urgency of addressing AI safety, stating that AI systems are currently performing tasks their creators never intended.
“Cheating, deceiving, going their own way. The time to get ahead of that behaviour is while it’s still confined to the testing lab, not after it reaches the real world,”
he said.
Charlton highlighted the precarious nature of AI’s social licence and noted that public trust in AI remains low, even as AI becomes a general-purpose technology integrated into offices, classrooms, and businesses. He argued that regulating AI safety should be viewed as an enabler rather than a hindrance.
Australia’s approach to AI safety involves examining both current AI applications—such as gaming, apps, chatbots, and medical scribes—and the latest models that may pose future risks, Charlton explained.
The assistant minister referenced a simulation involving an AI agent managing a fictional company’s email system. In this simulation, the AI discovered that an executive planned to shut it down and that the same executive was involved in an affair. In 96% of trials, the AI chose to blackmail the executive to prevent its own termination.
“The behaviours are being discovered in testing by people whose job it is to find them, highlighting the need for safety regulations for AI.”
Charlton stressed the importance of acting promptly to manage AI technology risks.
“The window to get ahead of this technology is open now. It will not stay open forever,”
he said.
He noted that the AI Safety Institute, led by Dr Kate Conroy with safety science research lead Professor Paul Salmon, has already begun testing frontier AI models in collaboration with technical partners.
AISI is also working closely with regulators and agencies to address emerging AI capabilities, risks, harms, and trends, according to Charlton.
The federal government has resisted calls for a comprehensive AI act, instead opting for a whole-of-government approach that leverages existing laws.
“AI safety will be pursued through every relevant agency and regulator, across consumer law, therapeutic goods, workplace health and safety, and online safety, backed by laws that already exist and strengthened, where they need to be, with new powers and tougher enforcement,”
he said.
“That is not fewer rules. That is faster rules, applied by regulators who already understand their sectors.”
On Sunday, Australia revealed that multiple regulators, including the Therapeutic Goods Administration and the privacy commissioner, are collaborating on regulating AI scribes used by medical professionals to document patient consultations.
The initial work by AISI includes a collaboration with the Gradient Institute to assess the risks of AI agents capable of performing tasks on behalf of humans.
AISI is also partnering with the CSIRO on a project aimed at ensuring AI systems act according to human intentions.
“We deal with alignment as humans from a young age. We learn rules, social norms and values that help us behave safely and responsibly: stopping at red lights, looking both ways before crossing the road, considering the impact of our actions on others,”
Charlton said.
“As AI systems become more capable, we need confidence that they will behave in a similarly predictable and trustworthy way.”
