19 Dec 2022 -
The relative costs of biological and non-biological intelligence pose an intriguing question. Let’s begin by exploring the costs associated with non-biological intelligence.
Chat GPT
A popular intelligent agent today is ChatGPT, a fine-tuned variant of GPT-3.5 with approximately 175 billion parameters.
A top-tier GPU, such as the NVIDIA A100, has an 80GB memory size and can generate a token in about 6ms for a 3 billion parameter model. With an MSRP of $32,000, roughly eight A100s are needed to store ChatGPT’s model and tokens. An 8-GPU server could generate around 15-20 words per minute.
For about $500,000 (50% GPU / 50% overhead), you can have a ChatGPT-level artificial intelligence available 24/7, as long as it remains powered.
Next generation chat
This figure demonstrates the relationship between model parameter size, training token size, and computing performance. Compute-optimal scaling predictions are based on the common assumption that models tend to be undertrained.
GPT-4 (OpenAI) is rumored to have only slightly more parameters than GPT-3 but 10 times the training data (tokens). Given that is priced at 10x the cost of GPT-3 the operating cost may very well be 10x too.
Human
Scientists estimate that the human brain contains an average of 86 billion neurons and 100 trillion synapses. Although there isn’t a direct comparison between parameters and neurons or synapses, parameters are generally believed to be equivalent to synapses.
In theory, human-level intelligence would require 100 trillion parameters and 1 petabyte of tokens for training.
- 6 times the parameters of ChatGPT
- 500 times the token training data estimated for GPT-4
It appears that the hardware required to achieve human-level intelligence, from a pure compute/parameter size perspective, could be reached by 2026 (+/- 3 years), or 2029 conservatively. It may take until 2035 for these artificial intelligences to be recognized as human-level, but by that time, they could already be significantly more intelligent than any human or group.
Two types of non-biological human-level intelligence may emerge: one less expensive to operate but with a high cognitive load when adapting to new situations, and another more expensive to operate but with a low cognitive load, making it more efficient at handling continuous changes.
However, some challenges remain:
- The compute elements (GPU, TPU, or others) may not have sufficient memory to store the model for several more years.
- Training data is limited, so new sources of data or training methods may be necessary.
Training Data Considerations
Human-level intelligence is estimated to require 100 trillion parameters and 1 petabyte of tokens for training. Some models suggest that there are 30 trillion words on the internet, but this may not be sufficient. Including images, videos, and audio in the training data could be an option.
The potential risk of having an AI use its knowledge to create content and train subsequent generations is also worth considering.
Humans can be effectively trained without 30 trillion words of training data, so it might be possible to train AI with less data as well.
What is a human likely to cost
A non-biolgical human intelligence is likely to cost $20M to $50M (GPU + system overhead). Of course an order of magnitude or more cost will need to be invested in design, training, and testing.
Conclusion
Is it more cost-effective to replace a human with an artificial intelligence? The answer depends on various factors. Non-biological intelligence has the advantage of never getting sick, working continuously, and being free from labor issues. However, a team of several humans could potentially cost the same as a single non-biological intelligence.
As technology continues to advance, the capabilities and intelligence of artificial systems will undoubtedly increase as well. The future remains uncertain, but it’s essential to consider the possibilities and implications of these developments.