There is a high probability of advanced forms of artificial intelligence, or AI, becoming “existentially dangerous to life on Earth”, scientists have claimed.
n an article in the academic journal AI Magazine, they warned of catastrophic consequences” if the development of certain artificial intelligence agents continues.
Researchers from Oxford University and Google’s artificial intelligence division DeepMind made the claims about artificial intelligence, which generally involves computers that are programmed to “think” for themselves.
Leading philosophers such as Oxford University’s Nick Bostrom have previously spoken of the threat posed by advanced forms of artificial intelligence, though one of authors of the new paper claimed such warnings did not go far enough.
One researchers, Michael Cohen, said Professor Bostrom, computer scientist Stuart Russell and others “have argued that advanced AI poses a threat to humanity”.
“Under the conditions we have identified, our conclusion is much stronger than that of any previous publication – an existential catastrophe is not just possible, but likely,” he wrote on Twitter thread accompanying the article.
The research paper proposes a scenario whereby an AI agent figures out a strategy to cheat to receive a reward that it is pre-programmed to seek.
In order to maximise its reward potential, it requires as much energy as is possible to obtain. The thought experiment sees humanity ultimately competing against the AI for energy resources.
“Winning the competition of ‘getting to use the last bit of available energy’ while playing against something much smarter than us would probably be very hard. Losing would be fatal,” Mr Cohen wrote.
“These possibilities, however theoretical, mean we should be progressing slowly – if at all – toward the goal of more powerful AI.”
DeepMind has already proposed a safeguard against such an eventuality, dubbing it “the big red button”.
In a 2016 paper titled Safely Interruptible Agents, it outlined a framework for preventing advanced machines from ignoring turn-off commands and going rogue.
Professor Bostrom previously described DeepMind – whose AI accomplishments include beating human champions at the board game Go and manipulating nuclear fusion – as the closest to creating human-level artificial intelligence.
He said AI could hold the potential to cure diseases and advance civilisation.
(© Independent News Service)