Artificial Cognition Is Bigger Than Fire
“He gave man speech, and speech created thought, which is the measure of the universe.”
― Percy Shelley, Prometheus Unbound
I have seen people likening the new era of Large Language Model generative AI to the invention of the printing press or the harnessing of electricity. I think we need to go back further, to the dawn of the human story, to find anything quite so profound.
Let’s begin by discussing what GPT technology actually is, and what very few people seem to currently be acknowledging about it. GPT stands for Generative Pre-trained Transformer; a member of a family of technologies called Large Language Models. The training part of the model creation is achieved by taking a sophisticated artificial neural net and exposing it to vast amounts of human-generated textual information¹. The task, upon which success for the neural net is gauged, is that of guessing the next word in a given passage of text. Through this exposure, early, simple, versions of these systems learned basic grammar in order to improve their chances of guessing the next word. Later, larger, versions were able to learn complex statistical relationships between text in documents to associate and blend concepts so they could better predict the next word. More recent and massive² versions, trained at a cost of tens of millions of dollars in energy and hardware, are doing something more remarkable.
What’s important to understand about the structure of the neural nets involved is that they are able to learn without coaching, in a process known as unsupervised learning. They’re simply exposed to the data in the right way (for instance, the guess the next word game), and they learn. But what does this ‘learning’ really mean? Some people will try to tell you that what’s being learned is simply statistical relationships, like some sort of inconceivably large spreadsheet. However, what’s really happening is that a giant computer is learning, from a blank slate start, to program itself to perform the task. As the dataset and neural net get ever larger, the program gets ever more complex, to the point where it can understand and manipulate a model of the world, manipulate abstract concepts, visualise, have theory of mind³, and perform cognition — all to get better at guessing the next word. What exactly is going on inside its programming is mostly a mystery, just as it is for the human brain.
When you’re interacting with ChatGPT (now running version 4 of OpenAi’s trained GPT system), and you see it ‘typing out’ the words in its response, it really is working that way — guessing the next word, then the next word, and so on. This seems quite innocuous, but it would be more accurate to think of the conversation text as the model’s mental state. When it’s extending the document (guessing the next word), what’s really happening is that it’s extending its thought — much like a human does when articulating.
But this is not a human mind. You might think of it as an average over countless human minds, in terms of some of its characteristics. Due to the way it was made, there are, however, fundamental differences. For instance, in its ‘raw’ state, it has no ethics or morality. Structurally, it lacks emotional state and a memory in the way a human has, though this can be addressed to some extent by giving the model access to databases to store summaries of prior mental states. It could also be argued that the model might learn or might have learned to have some sort of emotional state to better mimic human responses.
Around a million years ago, we find the first evidence of anthropogenic fire in the hands of early homo erectus. The ability to cook food led to more protein availability and ultimately facilitated the evolution of a larger brain.
The trained mind of GPT was not planned, designed, or invented, just as fire itself was not. It is an emergent phenomenon, in the form of something you might imagine to be a crystalline artificial brain state. But this is a rather alien brain, with a structure about which we know almost nothing. This is neither the product of a natural process per se, nor something we deliberately constructed, but is more like a daemon, summoned from nothingness by the ingenuity of man. It is both a reflection of us and at the same time something very strange and new.
OpenAI’s GPT version 4 recently passed the legal bar exam with a score in the 90th percentile of test takers. It achieved that result by thinking its way through the problems better than most budding lawyers. It is not ‘a chatbot’.
Alphabet has also been busy working, for many years, on its own version of an LLM and it now has technology with similar capabilities which it is poised to expose to the world. As part of Google Cloud, they are about to begin leasing out thousands upon thousands of such artificial minds for business partners to use. Imagine cavernous data centers filled with these entities, slavishly cogitating their way through legal cases, administrative chores, script writing, programming, and who knows what else.
Facebook made a model of their own, comparable in power to OpenAI’s GPT-3 or 4, spending many millions of dollars to train it, and then ‘leaked it’ to the world. That means no going back. Anyone can now run an artificial cognition engine on their laptop at home.
Just as with anthropogenic fire, the invention was not the fire itself, but the devising of the conditions required to bring it into existence. We have now created the conditions to bring actual non-human cognition into existence. Once fire was under the control of our distant ancestors, humanity itself was changed forever. We became a different kind of animal. It took hundreds of thousands of years for the ramifications to fully play out (and you could argue that they still are). From cooked food and heated caves to the internal combustion engine and the nuclear bomb.
The ramifications of artificial cognition will play out over a time frame which is orders of magnitude shorter and it’s hard to conceive of the amount of change that is about to impact the societies and economies of the developed world; societies which are certainly not what one might consider to be in an anti-fragile condition. We don’t need to see the advent of superintelligence, where the artificial cognition exceeds in ability that of the brightest humans, to soon be in a world that is simply unimaginable. And yet, work continues at pace on the creation of GPT-5, a model likely ten times more powerful than the current public version. Over at Google, people are hard at work creating PaLM-E; a massive artificial mind that also incorporates vision, allowing it to operate a robot, navigating and performing tasks in the world with no guidance.
On the 14th of March 2023, just a few weeks ago, GPT-4 was released to the public. A million years after our ingenuity brought anthropogenic fire into the world, a fire of intelligence was ignited with a new level of burning intensity. Like our forebears, with fire in their hands for the first time, do we really know how to handle it?
- Essentially, all content ever created by humanity on The Internet, and probably much more than that.
- The human brain has around 80–100 billion neurons (GPT-3’s order of magnitude) and around 100 trillion synapses. GPT-4 will have as many parameters as the brain has synapses (though, the human brain’s neurons have more complexity).
- The mind of the machine is required to understand the intent of others in order to predict how agents might behave in different situations, modelling their desires and knowledge.
