ChatGPT Maker OpenAI , announced on Wednesday that it would devote significant resources to the effort and establish a new research team to examine how to make artificial intelligence that eventually controls itself safe for people.
Ilya Sutskever, a co-founder of OpenAI, and Jan Leike, the director of alignment, wrote in a blog post that “the vast power of superintelligence could… lead to the disempowerment of humanity or even human extinction.” We currently lack a method for directing or managing a potentially superintelligent AI and preventing it from acting erratically.
Superintelligent AI – systems more intelligent than humans – could arrive this decade, the blog post’s authors predicted. Humans will need better techniques than currently available to be able to control the superintelligent AI, hence the need for breakthroughs in so-called ”alignment research,” which focuses on ensuring AI remains beneficial to humans, according to the authors.
ChatGPT Maker OpenAI , backed by Microsoft, is dedicating 20% of the compute power it has secured over the next four years to solving this problem, they wrote. In addition, the company is forming a new team that will organize around this effort, called the Superalignment team.
The team’s goal is to create a ”human-level” AI alignment researcher, and then scale it through vast amounts of compute power. OpenAI says that means they will train AI systems using human feedback, train AI systems to assistant human evaluation, and then finally train AI systems to actually do the alignment research.
AI safety advocate Connor Leahy said the plan was fundamentally flawed because the initial human-level AI could run amok and wreak havoc before it could be compelled to solve AI safety problems.
