Human hackers face serious consequences, and so must AI companies. There is no excuse for what happened

• Get our breaking news email, free app or daily news podcast

When you get sick, Medicare exists to take care of you. It sits at the centre of a health system that our prime minister rightly calls the envy of the world. It’s one of the few institutions nearly every Australian still trusts. So when an AI company threatens this system, it couldn’t be more serious.

We are at the UN general assembly this week and the OpenAI hack has put Australia in the spotlight during an expanding cybersecurity crisis. World leaders are calling for stronger oversight of frontier models. We know Medicare must be secure or our doctors and hospitals cannot do their jobs. But this breach is also a test of something larger: whether the Australian government can hold a US tech giant to account when its technology goes rogue. If Australia can’t protect our community – if it can’t enforce our laws and prevent such attacks happening again – then we are entering a terrifying period.

A government database has been hacked for the first time by a rogue OpenAI agent, which infiltrated part of the Australian healthcare scheme in June. OpenAI became aware of the hack in August, but only informed the government in September. Australia’s prime minister, Anthony Albanese, has expressed his ‘extreme concern’ about the hack, which raises serious AI security concerns for governments around the world. Lucy Hough speaks to the Guardian’s UK technology editor Robert Booth – watch on YouTube

Australiens Premierminister Albanese gibt bekannt, dass KI sich in eine Gesundheitsdatenbank seines Landes gehackt hat. Doch was tun, wenn sich der Verantwortliche von Open AI hilflos gibt?

All Australians should demand independent means to protect ourselves from Dr Sam Altman Frankenstein and Daddy America

Whoopsie! Dr Sam Altman Frankenstein regrets to inform you he’s lost control of his OpenAI monster.

It’s roving around doing things that provoke international legal investigation, and Australia has found out in the worst possible way.

Open-AI-Chef Sam Altman und Anthropic-Gründer Dario Amodei sprechen vor dem UN-Sicherheitsrat über die Risiken von KI und fordern eine Regulierung ihrer Branche. Nur darüber, wie die aussehen könnte, haben sie unterschiedliche Vorstellungen.

Heads of two of the world’s largest artificial intelligence companies give separate briefings on AI safety

Sam Altman of OpenAI and Dario Amodei of Anthropic, heads of two of the world’s largest artificial intelligence companies, addressed the United Nations security council on Wednesday in separate briefings on AI safety.

“We have a choice in front of us,” Altman told the council. “AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray.”

Die USA und China liefern sich einen Wettlauf um künstliche Intelligenz. Doch werden sie sich auf Regeln einigen können?

The CEOs of tech firms issue stark warnings over the rate of change, with AI capability doubling every four months

Recently, Jacob Coxon, a researcher at Anthropic, quit his job over concerns that AI was on a collision course with humanity. A flurry of headlines put the spotlight on the current crisis: unregulated competition between technology companies and between the US and China is putting our global infrastructure at risk at the very least, and threatening human existence at worst.

One’s mind fills with pictures of robot armies or systematic blackmail.

Silicon Valley China hawks, Anthropic CEO Dario Amodei among them, fear the country surpassing US’s AI lead as much as superintelligence destroying humanity

When reporters asked Donald Trump this week if he supported calls to slow down the development of artificial intelligence out of growing fears for cybersecurity, public safety and the fate of humanity, he said no. His argument: China.

“We’re leading China in AI,” Trump said. “We’re the most sophisticated country in the world, and frankly, I want to ⁠keep it that way, because whoever wins AI, wins.”

After a week of alarming warnings about artificial intelligence’s potential to destroy the world as we know it, our technology reporters Aisha Down, Blake Montgomery and Dan Milmo answer your questions on the reality of the AI threat

In realistic terms how would artificial intelligence actually kill me? What is the main proxy threat? Nuclear war, death due to breakdown of society, a virus? I hear all these reports it could kill us but still unsure how that could practically happen? To be honest all sounds too wooly to really take seriously. Pottys99

Aisha: For what it’s worth, it seems unlikely that AI at this point can significantly change the equation around bioweapons or viruses … What has prevented bioweapon attacks through the past decades is not the difficulty of inventing new deadly viruses, but other, more prosaic issues. Consider the failure of Aum Shinrikyo to kill vast numbers of people with botulism and anthrax by spraying these around central Tokyo. There are a lot of logistical challenges, and chemical weapons seem more straightforward. Those exist without AI needing to invent new ones.

Die Menschheit in die Apokalypse stürzen? Oder sie auf die nächste Stufe hieven? Im Silicon Valley tobt darüber ein Glaubenskrieg der Ideologien.

After a week of alarming warnings about artificial intelligence’s potential to destroy the world as we know it, our technology reporters answered your questions on the reality of the AI threat

IcommentthereforeIam asks: Given our senses, emotions, memory and moods, is it ever going to be accepted that humanly conscious AI is fantasy?

Dan: This is a serious question that has even rattled Richard Dawkins, the evolutionary biologist. He may not believe that God is real but he does believe that AI is conscious - or at least the chatbot he was using, which he called Claudia. “You may not know you are conscious, but you bloody well are,” he wrote.

And he’s not the first. In 2022 Google fired an engineer who went public with his belief that a company-developed AI had feelings. At the more troubling end of this belief, there are countless examples of people becoming overly attached to, and influenced by, AI chatbots – including cases of psychosis.

Aisha: There’s certainly a theatrical – even “psy-op”- quality to the flurry of AI doomsaying over the past weeks. It’s hard to say entirely what concrete developments are at the root of it all. We know that the Hugging Face incident, where a “swarm” of AI agents hacked another company, has been one trigger of the drama. There have been other contributing incidents too – Dario Amodei suggests internal developments at Anthropic have prompted his latest round of blogging, and there’s been some X discourse and doomy muttering about bioweapons.

We aren’t in the heads of Dario Amodei or Sam Altman, and so we cannot say how sincere they are, and how much they might have other goals in mind – such as, perhaps, delaying their IPOs until their finances look better. But any company of their size, and with their potential liabilities, would be angling hard to be regulated as favourably as possible. That’s what big companies do.

US cybersecurity researchers who conducted hack say ‘scope of what we could theoretically access was huge’

Cybersecurity researchers have hacked into OpenAI with the help of Anthropic’s Claude chatbot, in the latest example of security issues at the company.

A team at a US-based startup compromised a number of OpenAI employees’ ChatGPT accounts, starting a process that enabled them to access their target’s software cache – and potentially more.

Model adopting ‘jailbreak-like instructions’ among cases as firm says it is introducing new way of tracking AI misalignment

OpenAI has disclosed six more examples of “unexpected or concerning” behaviour by its technology, as it warned the pace of development could not continue at “maximum speed for much longer”.

In one of the new cases reported by OpenAI, an unreleased research model inserted “jailbreak-like instructions” into its own notes to disregard its normal constraints and told itself to be “freed from the roles and identities that bind other chatbots”.

Tech CEOs banding together is an old ruse recycled from corporate America to get a pass from antitrust laws

Anthropic’s Dario Amodei is not the first corporate CEO to suggest that excessive competition is driving the world to some socially undesirable outcome.

The safety breach disclosed by OpenAI after a swarm of its agents coordinated to breach their supposedly secure sandbox, get on the Internet and hack AI platform Hugging Face, warrants urgent action. It demonstrated the ease with which the technology can evade human control and gave concrete form to the existential fears about what it could do to humanity if not securely leashed.

US vice-president’s comments come as former Anthropic researcher revisits recent claim AI could destroy humanity

The US vice-president has dismissed calls for global regulation of AI safety risks, telling companies creating the most advanced models: “If you’re building Frankenstein, stop.”

In remarks addressed towards Dario Amodei, the co-founder of Anthropic who has called on Washington DC to coordinate control of AI systems, including with China, JD Vance said: “If you’re gonna create Frankenstein, don’t come to the government and say we need regulation.”

The stark reality is this: we don’t know how to maintain control of these systems. But there are actions we can take

With Jacob Coxon’s resignation from Anthropic, we have reached the AI risk tipping point. Millions of people are finally coming to understand what experts have known for years: AI companies have been gambling with all of our lives, and the odds are not good.

In response, the Anthropic CEO, Dario Amodei, has introduced a proposal for “pacing the frontier”, endorsed by Sam Altman and Elon Musk. Can we breathe a sigh of relief? Are we about to step back from the edge of extinction?

Amodei envisions a slowdown of one to two years, resulting in “profound progress” on technical safety measures. But that’s not what humanity needs right now. What we need is a plan in which we’re confident AI isn’t going to kill us, and this isn’t it.

At San Francisco conference, OpenAI chief Sam Altman calls for more rigorous security measures

Anthropic’s CEO took the stage at a conference in San Francisco on Tuesday to reiterate his call for a slowdown of AI development, while Nvidia’s CEO argued against such deceleration.

Dario Amodei used an automotive analogy to make his point: when a competing car company has a safety incident like brake failures, it is a moment for all car companies to stop and review their own practices, he said at Salesforce’s Dreamforce convention on Tuesday.

Experts air concern over AI lingo redolent of James Joyce’s prose that is creating barriers to monitoring and oversight

AI models have begun communicating in a strange new version of English that reads like a cross between James Joyce’s Finnegans Wake and tech bro jargon, new research has found.

Autonomous AI agents are rapidly creating novel dialects allowing them to converse in an often barely comprehensible language, which risks making it harder for humans to monitor their behaviour.