Cost of intelligence

Cost of intelligence Joe Zott 19 Dec 2022 -

An interesting question is the relative cost of biological and non-biological intelligence. Starting with the question on how expensive is non-biological intelligence.

Chat GPT

Today a popular “intelligent” agent is the Chat GPT.

ChatGPT is supposed to be a fine tuned varient of GPT-3.5 so it is likely to have about 175B parameters.

A leading edge GPU is the NVIDIA A100 with a memory size of 80Gb. A single NVIDIA A100 capable of storing a 3B parameter model can generate a token in about 6ms. It has an MSRP of $32K.

About 8 A100s appear to be needed to store the ChatGPT model and tokens. Such an 8 GPU server could generate on the order of 15 - 20 words a minute.

So for about $1/2M (50% GPU / 50% overhead) you can have a Chat GPT level non-biological intelligence available 365 days a year/ 24 hours a day as long as you keep it powered.

Next generation chat

I found this figure that shows the relationship between model parameter size, trainining token size, and computing performance. The prediction of compute-optimal scaling is baseed on common estimates that models have tended to be undertrained.

parameter token

GPT-4 (openai) is rumored to be only slightly larger (parameters) than GPT-3, but to have 10x the training data (tokens). So while more expensive to train, it should have similar operating costs.


Researchers estimate that the human brain contains an average of 86 billion neurons and 100 trillion synapses. There isn’t a good model between parameters and neurons or synapses, but generally parameters are believed to be equivalent to synapses.

So theoretically human level intelligence requires 100T parameters and 1P tokens for training.

model year

So it looks like the hardware to achieve human level intelligence can be reached (from a pure compute / parameter size) in about 2026 (+/-3 years) - so 2029 to be conservative. I am guessing that it may take until 2035 before we declare these as human level intelligence, but my estimate is that by this time the non-biological will be effectively much smarter than any human (or group).

One interesting observation that I made in evaluating the implementation innovation is the rise of two classes of non-biological human level intelligence. One less expensive to operate will efficiently adapt to new situations and dealing with them will represent a high cogntive load. The other will be more expensive to operate, but will be able to handle, adapt to, and optimize to new situations with a low cognitive load - so able to handle continual changes.

But a few gaps will remain:

Where does the training data come from

As I mentioned human level is likely to require 100T parameters and 1P tokens for training. Some models suggest that there are 30T words on the internet. Not sure where the training data would come from as there is not enought Internet content to train on just Internet data. Certainly images, video, and audio has to be added to the available to learn from.

There is the risk of having an AI train use its knowledge to create content and train subsequent generations, but perhaps that might be a direction to go.

We are able to effectively train humans without 30T words of training data. So perhaps we can train an AI without 30T words of training data.

What is a human likely to cost

A non-biolgical human intelligence is likely to cost $20M to $50M (GPU + system overhead). Of course an order of magnitude or more cost will need to be invested in design, training, and testing.


Is it cheaper to replace a human with a golem? It will depend. Certainly the non-biological intellence never gets sick, works continually, and has no labor issues. Balancing that is that is that a team of half a dozen people may easily be equal in cost to a single non-biological intelligence.

Of course technology continues to advance and the capabilities and intelligence of non-biological intelligence will continue to rise.

To be seen what the future holds.