{"id":57235,"date":"2023-11-27T18:58:03","date_gmt":"2023-11-27T18:58:03","guid":{"rendered":"https:\/\/entertainment.runfyers.com\/index.php\/2023\/11\/27\/contrary-to-reports-openai-probably-isnt-building-humanity-threatening-ai-techcrunch\/"},"modified":"2023-11-27T18:58:03","modified_gmt":"2023-11-27T18:58:03","slug":"contrary-to-reports-openai-probably-isnt-building-humanity-threatening-ai-techcrunch","status":"publish","type":"post","link":"https:\/\/entertainment.runfyers.com\/index.php\/2023\/11\/27\/contrary-to-reports-openai-probably-isnt-building-humanity-threatening-ai-techcrunch\/","title":{"rendered":"Contrary to reports, OpenAI probably isn’t building humanity-threatening AI | TechCrunch"},"content":{"rendered":"


\n<\/p>\n

\n

Has OpenAI invented an AI technology with the potential to \u201cthreaten humanity\u201d? From some of the recent headlines, you might be inclined to think so.<\/p>\n

Reuters<\/a> and The Information<\/a> first reported last week that several OpenAI staff members had, in a letter to the AI startup\u2019s board of directors, flagged the \u201cprowess\u201d and \u201cpotential danger\u201d of an internal research project known as \u201cQ*.\u201d This AI project, according to the reporting, could solve certain math problems \u2014 albeit only at grade-school level \u2014 but had in the researchers\u2019 opinion a chance of building toward an elusive technical breakthrough.<\/p>\n

There\u2019s now debate as to whether OpenAI\u2019s board ever received such a letter \u2014 The Verge cites a source<\/a> suggesting that it didn\u2019t. But the framing of Q* aside, Q* in actuality might not be as monumental \u2014 or threatening \u2014 as it sounds. It might not even be new.<\/p>\n

AI researchers on X (formerly Twitter) including Yann LeCun, Meta\u2019s chief AI scientist Yann LeCun, were immediately skeptical that Q* was anything more than an extension of existing work at OpenAI \u2014 and other AI research labs besides. In a post on X, Rick Lamers, who writes the Substack newsletter Coding with Intelligence, pointed<\/a> to an MIT guest lecture OpenAI co-founder John Schulman gave seven years ago during which he described a mathematical function called \u201cQ*.\u201d<\/p>\n

Several researchers believe the \u201cQ\u201d in the name \u201cQ*\u201d refers to \u201cQ-learning,\u201d an AI technique that helps a model learn and improve at a particular task by taking \u2014 and being rewarded for \u2014 specific \u201ccorrect\u201d actions. Researchers say the asterisk, meanwhile, could be a reference to A*, an algorithm for checking the nodes that make up a graph and exploring the routes between these nodes.<\/p>\n

\n
\n

Please ignore the deluge of complete nonsense about Q*.
One of the main challenges to improve LLM reliability is to replace Auto-Regressive token prediction with planning.<\/p>\n

Pretty much every top lab (FAIR, DeepMind, OpenAI etc) is working on that and some have already published\u2026<\/p>\n

\u2014 Yann LeCun (@ylecun) November 24, 2023<\/a><\/p>\n<\/blockquote>\n<\/div>\n

Both have been around a while.<\/p>\n

Google DeepMind applied Q-learning to build an AI algorithm that could play Atari 2600 games at human level\u2026 in 2014. A* has its origins in an academic paper published in 1968. And researchers at UC Irvine several years ago explored<\/a> improving A* with Q-learning \u2014 which might be exactly what OpenAI\u2019s now pursuing.<\/p>\n

Nathan Lambert, a research scientist at the Allen Institute for AI, told TechCrunch he believes that Q* is connected to approaches in AI \u201cmostly [for] studying high school math problems\u201d \u2014 not destroying humanity.<\/p>\n

\u201cOpenAI even shared work earlier this year improving the mathematical reasoning of language models with a technique called process reward models,\u201d Lambert said, \u201cbut what remains to be seen is how better math abilities do anything other than make [OpenAI\u2019s AI-powered chatbot] ChatGPT<\/a> a better code assistant.\u201d<\/p>\n

Mark Riedl, a computer science professor at Georgia Tech, was similarly critical of Reuters\u2019 and The Information\u2019s reporting on Q* \u2014 and the broader media narrative around OpenAI and its quest toward artificial general intelligence (i.e. AI that can perform any task as well as a human can). Reuters, citing a source, implied that Q* could be a step toward artificial general intelligence (AGI). But researchers \u2014 including Riedl \u2014 dispute this.<\/p>\n

\u201cThere\u2019s no evidence that suggests that large language models [like ChatGPT] or any other technology under development at OpenAI are on a path to AGI or any of the doom scenarios,\u201d Riedl told TechCrunch. \u201cOpenAI itself has at best been a \u2018fast follower,\u2019 having taken existing ideas \u2026 and found ways to scale them up. While OpenAI hires top-rate researchers, much of what they\u2019ve done can be done by researchers at other organizations. It could also be done if OpenAI researchers were at a different organization.<\/p>\n

Riedl, like Lambert, didn\u2019t guess at whether Q* might entail Q-learning or A*. But if it involved either \u2014 or a combination of the two \u2014 it\u2019d be consistent with the current trends in AI research, he said.<\/p>\n

\u201cThese are all ideas being actively pursued by other researchers across academia and industry, with dozens of papers on these topics in the last six months or more,\u201d Riedl added. \u201cIt\u2019s unlikely that researchers at OpenAI have had ideas that have not also been had by the substantial number of researchers also pursuing advances in AI.\u201d<\/p>\n

That\u2019s not to suggest that Q* \u2014 which reportedly had the involvement of Ilya Sutskever, OpenAI\u2019s chief scientist \u2014 might not move the needle forward.<\/p>\n

Lamers asserts that, if Q* uses some of the techniques described in a paper<\/a> published by OpenAI researchers in May, it could \u201csignificantly\u201d increase the capabilities of language models. Based on the paper, OpenAI might\u2019ve discovered a way to control the \u201creasoning chains\u201d of language models, Lamers says \u2014 enabling them to guide models to follow more desirable and logically sound \u201cpaths\u201d to reach outcomes.<\/p>\n

\u201cThis would make it less likely that models follow \u2018foreign to human thinking\u2019 and spurious-patterns to reach malicious or wrong conclusions,\u201d Lamers said. \u201cI think this is actually a win for OpenAI in terms of alignment \u2026 Most AI researchers agree we need better ways to train these large models, such that they can more efficiently consume information<\/p>\n

But whatever emerges of Q*, it \u2014 and the relatively simple math equations it solves \u2014 won\u2019t spell doom for humanity.<\/p>\n<\/p><\/div>\n