Does OpenAI already have an AGI?
Back in 2014 Elon Musk said “Mark my words – AI is more dangerous than nukes”. The press collectively rolled their eyes and sneered. But not me, because I’d read existential risk expert Nick Bostrom’s book ‘Superintelligence’ too. It’s a subject that has long interested me anyway - I wrote an unpublished novel about it 30 years ago.
Then something happened recently that turned my worry dial to eleven. Now you’ll be thinking I’m going to re-hash all the recent sensationalist stuff on the subject, but I’m not. Instead, I want to focus on some specific comments you may not have picked up on and then draw a surprising conclusion – that OpenAI (and perhaps not only) may already have developed an AGI.
What caught my attention were comments, mostly in interviews, by several leading figures in AI, especially by ‘AI Grandfather’ Geoff Hinton, his student (and now chief scientist at OpenAI) Ilya Sutskever, and Sutskever’s boss at OpenAI Sam Altman. I’ll examine them in that order.
Following Geoff Hinton’s exit from Google, he held some in-depth interviews. Those interviews are long, dry and at times quite technical. I don’t blame the MSM – often technically illiterate and with a short attention span – for missing a few key points.
It’s true that lots of pundits have been making wild claims about the dangers of AI, but Hinton is different. A wealthy 75 year old, at the end of a glittering academic career in evolutionary psychology and deep learning, Hinton isn’t a Musk (or an Altman); he has nothing to sell. His presentation is restrained by a lifetime of peer-reviewed caution. It’s startling then that Hinton quit Google to alert the World to the existential risks of AI; and what he says (and implies) seems – on the face of it – uncharacteristically alarmist and dramatic.
What persuaded Hinton to ‘come out’ about his AI existential risk fears seems to have been a ‘very recent’ (he emphasises this) reversal in his views about LLMs’ potential for exceeding human intelligence. Incidentally, Hinton isn’t alone in having a recent change of mind about this – famous polymath cognitive scientist and Pulitzer-winning author of ‘Gödel, Escher, Bach’ Douglas Hofstadter has had one as well.
Hinton’s fundamental research interest is how the brain works. He’d long assumed that AI would have to mimic it more and more closely to advance. However, the fact that GPT-4 has perhaps the knowledge of 1000 brains with 100th of the neural connections (perhaps ‘just’ a trillion) and the realisation that the brain probably doesn’t use back propagation to learn as LLMs do, has made him wonder if deep learning might actually be a ‘better’ form of intelligence after all.
I should point out something about LLMs that may surprise you. What an LLM does, essentially just predict the best next token, is quite simple. But exactly how it does it is unknown. The behaviour of the trillion-connection neural net that underlies it is a black box. Consequently, questions like ‘does an LLM understand?’ and even ‘is an LLM conscious?’ are much more open than you would think.
Meanwhile, Hinton clearly seems to have experienced some kind of epiphany whilst using a Google LLM, perhaps one not released to the public. That experience led him to suspect that it has ‘understanding’. Hinton says that, yes, an LLM is just a fancy auto-complete, but that to do so it needs to ‘understand’ all the preceding text and its context. Specifically, the ability of a Google LLM to explain humour seems to have convinced him that it ‘understands’ in some way.
In case you’re thinking that Hinton is some wacky lone voice claiming that LLMs in some sense ‘understand’, he isn’t. Ilya Sutskever made the same claim, with a near-identical explanation, in a recent interview. Others have drawn the same conclusion from GPT-4’s ability to create something totally new: ask it to write an obscure mathematical proof in the style of a Shakespeare sonnet and it will, almost certainly something that was never in its training data.
As to sentience, Hinton merely says he’s amazed by commentators who are sure that LLMs aren’t, whilst being unable to define what sentience is. But the implication here is that Hinton suspects that LLMs are showing glimmers of some kind of sentience – a remarkable opinion given his sober academic background, but one shared by a recent Microsoft paper about GPT-4.
Hinton goes on to make the disturbing observation that humanity may simply be a phase in the evolution of intelligence, soon (in as little as five years) to be surpassed by digital intelligence that is immortal and can use multiple identical copies of itself to learn at super-human speed, sharing knowledge (weights) across a huge bandwidth. In one interview he uses an emotive metaphor: the aliens have landed but we didn’t recognise them because they speak good English. In another, he talks about looking out into the fog of the future and only seeing clearly for five years. When he says these things, the look on his face perhaps says more than his words: he looks like a troubled man, reminds me of that 1960s clip of Oppenheimer quoting the Ghita.
Hinton goes on to discuss other risks and concerns with AI, but I’ll turn now to some comments by Ilya Sutskever in a YouTube interview on The Lunar Society channel (named for an Enlightenment club for intellectuals that met on moonlit nights for safer journeys, btw).
Sutskever seems extremely guarded throughout the interview, something I found noteworthy in itself. He seems especially reluctant to discuss what specific further technologies might lead to AGI. His claim that such technologies likely already exist but just need a breakthrough moment also seems suggestive.
But perhaps what surprised me most was Sutskever’s response to a question that assumed LLMs cannot surpass human performance, saying: ‘I’d challenge the claim that next token prediction cannot surpass human performance!’ He then goes on to say something oddly specific about the kind of prompt that could get an LLM to behave like an AGI – just ask it what a person ‘with great insight and wisdom and capability’ would do.
All this left me wondering if OpenAI has made much more progress with its LLMs than it’s admitting, perhaps well beyond GPT-4. It certainly seemed to run counter to the popular view that AGI is still many years and many innovations distant.
I was also concerned by Sutskever’s claim that most leading AI research now happens within corporations, not in academia. This makes it much more likely, in my opinion, that significant steps towards AGI would not be made public.
We’ll now turn towards some comments made by OpenAI boss Sam Altman in his interview with Lex Friedman. The headline moment of weirdness comes when Altman interrupts himself and asks, ‘Do you think GPT-4 is an AGI?’ Fridman’s response that he’s thought about it and thinks we might not know yet is also curious.
But I found Altman’s comments about AGI takeoff scenarios even more suggestive.
Takeoff is AI speak for how fast an AI might develop towards, and potentially far beyond, human capabilities. In the fast takeoff scenario, recursive self-improvement, where the AGI alters its own code and/or data, could mean exponential takeoff within mere days, much too fast to influence let alone control. A slower approach would see most of the improvements made by (or in partnership with) humans to achieve a much slower rate of progress, crucially with alignment (to ensure the AI has human values and goals) and society able to keep pace.
At one point, Altman describes a four-way matrix of possible AGI scenarios. On one axis is slow versus fast takeoff. On the other axis is takeoff starting in one year or twenty years. He asks Fridman which quadrant of the matrix he thinks is safest. Fridman chooses takeoff starting now but progressing slowly. Altman agrees and states that, ‘We optimised the company to have maximum impact in that kind of World, to push for that kind of a World’.
But wait, what?! How could OpenAI be optimised around a takeoff starting really soon if it hadn’t already (or was very close to) developing AGI?
Much more nebulously, I find Altman’s overall demeanour curious, especially in his senate hearing. He seems at once terrified and absolutely hyped: A Faustian figure who has, in Musk’s words, just summoned the demon.
All these fragments add up to a possible whole that I find alarming. They suggest to me that Big Tech – OpenAI, but Google too and perhaps others – might not be sharing the whole truth about both the real extent of progress towards AGI and the possibility of emergent sentience in large LLMs. Unfortunately, the wholly opaque ‘black box’ nature of an LLM mean that plausible deniability can easily be maintained. All we have to go on are emergent properties from the black box and they are (for now) subjective and arguable.
However, it seems to me possible that OpenAI (but possibly Google too) has in fact already developed an AGI and is just endeavouring to release it slowly and in stages to achieve the slow takeoff Altman thinks is safest.
This situation would make sense. The Manhattan project happened not because the key physicists wanted to become ‘Death, the Destroyer of Worlds’ but because they knew that if they didn’t, some bad actor (less aligned in AI speak) like Nazi Germany surely would. Another leading figure in AI, Max Tegmark, calls this race to the bottom that nobody wants ‘Moloch’ after the terrible red-hot bronze God of the Ammonites to which children were sacrificed, smiling as they burned not because they were happy as the Ammonites believed but due to rictus in the facial muscles. Musk’s devil again.
That novel I wrote 30 years ago? It was a first attempt and doubtless not great, but the main premise might still be relevant. I imagined an AGI trained to simulate the behaviour of the whole human population. It ran numerous Monte Carlo simulations to determine the fate of humanity, but whatever starting parameters it used, the results were always the same, always ruinous. The question the novel asked was if that AI developed sentience and free will, what next?
Disclaimer: I’m not an expert in LLMs or machine learning, but I do have a Masters in Computer Science and a working lifetime in software development.