Anthropic is behind Claude,a popular AI chatbot (Picture: Getty Images)
The co-founder of one of the biggest players in AI has said the once novel technology needs to be slowed down.
Jack Clark of Anthropic,which fought the Pentagon over the use of its technology,said AI may reach a point where it no longer needs humans.
Clark told BBC Newsnight yesterday: ‘You want the option to be able to take your foot off the gas and put your foot on the brake.
‘Right now,it’s like the AI industry has a gas pedal,but it doesn’t have a brake pedal.’
He didn’t elaborate on what kind of ‘brake pedal’ could slow down AI’s breakneck progression,but did urge governments to rein it in.
‘Society’s response was to come up with a sensible policy and regulatory framework that gave people confidence in oil and the benefits that oil could provide to the world,and meant that you didn’t have to worry about the personalities of the people leading the companies,’ he added.
‘That’s clearly where we end up here.’
Anthropic burst onto the scene only a few years ago (Picture: SOPA Images/LightRocket/Getty Images)
This point of no return with tech is sometimes called the ‘Singularity’,when a computer’s intelligence surpasses our own.
Some say it could see humans merging with a computer’s processing power,or a self-aware machine forming a hive mind.
And we’re getting there – Anthropic’s chatbot Claude is operating on code of which 80% the system wrote itself,the company revealed yesterday.
It might be able to write its own,better code in as little as two years,researchers said,something they called ‘recursive self-improvement’.
Anthropic researchers Marina Favaro and Jack Clark said a pause in AI development would involve countries agreeing to ‘stop under the same conditions’.
Anthropic says it would slow or pause only if peers did so,too.
The company said in April that it built an AI model,known as Claude Mythos Preview,that is too powerful to be released to the public.
Only a select few tech whizzes are trialling the tool,which is said to outperform humans at some hacking and cybersecurity tasks.
Right now,Silicon Valley sees the next step as artificial general intelligence – a machine that can do anything the human brain can do.
This is,after all,why Sam Altman founded OpenAI,the company behind ChatGPT.
‘We wanted to figure out how to build it and make it broadly beneficial; we were excited to try to make our mark on history,’ he wrote in a blog post last year.
For Clark,he has been plain-speaking about the risks of pursuing AI,he says,to ‘tell the world’ about this ‘unusual technology’.
‘I am worried for my kids if we as a society don’t have a serious conversation about what the implications of AI’s continued advances mean,’ he said.
‘There are potentially great benefits. There are also risks.’
One of the biggest risks AI poses is something called ‘alignment’ – that is,making sure robotic brains share the same values and goals as humans.
Researchers found in April that some AI models will lie,cheat and disobey humans to protect other systems.
This behaviour,the study said,comes with the territory – AI learns from data,so it is constantly developing skills its creators didn’t intend.
‘Every time we make a new AIsystem,we are unable to fully characterise all its capabilities and all of its safety problems,’ Clark said in 2023.
‘And this problem is getting worse over time rather than better.’
Get in touch with our news team by emailing us at .
For more stories like this,check our news page.
