Superintelligence: Paths, Dangers, Strategies (13 page)

A second way to illustrate the concept of quality superintelligence is by noting the domain-specific cognitive deficits that can afflict individual humans, particularly deficits that are not caused by general dementia or other conditions associated with wholesale destruction of the brain’s neurocomputational resources. Consider, for example, individuals with autism spectrum disorders who may have striking deficits in social cognition while functioning well in other cognitive domains; or individuals with congenital amusia, who are unable to hum or recognize simple tunes yet perform normally in most other respects. Many other examples could be adduced from the neuropsychiatric literature, which is replete with case studies of patients suffering narrowly circumscribed deficits caused by genetic abnormalities or brain trauma. Such examples show that normal human adults have a range of remarkable cognitive talents that are not simply a function of possessing a sufficient amount of general neural processing power or even a sufficient amount of general intelligence: specialized neural circuitry is also needed. This observation suggests the idea of
possible but non-realized cognitive talents
, talents that no actual human possesses even though other intelligent systems—ones with no more computing power than the human brain—that did have those talents would gain enormously in their ability to accomplish a wide range of strategically relevant tasks.

Accordingly, by considering nonhuman animals and human individuals with domain-specific cognitive deficits, we can form some notion of different qualities of intelligence and the practical difference they make. Had
Homo sapiens
lacked (for instance) the cognitive modules that enable complex linguistic representations, it might have been just another simian species living in harmony with nature. Conversely, were we to
gain
some new set of modules giving an advantage comparable to that of being able to form complex linguistic representations, we would become superintelligent.

Superintelligence in any of these forms could, over time, develop the technology necessary to create any of the others. The
indirect reaches
of these three forms of superintelligence are therefore equal. In that sense, the indirect reach of current human intelligence is also in the same equivalence class, under the supposition that we are able eventually to create some form of superintelligence. Yet there is a sense in which the three forms of superintelligence are much closer to one another: any one of them could create other forms of superintelligence more rapidly than we can create any form of superintelligence from our present starting point.

The
direct reaches
of the three different forms of superintelligence are harder to compare. There may be no definite ordering. Their respective capabilities depend on the degree to which they instantiate their respective advantages—
how
fast a speed superintelligence is,
how
qualitatively superior a quality superintelligence is, and so forth. At most, we might say that,
ceteris paribus
, speed superintelligence excels at tasks requiring the rapid execution of a long series of steps that must be performed sequentially while collective superintelligence excels at tasks admitting of analytic decomposition into parallelizable sub-tasks and tasks demanding the combination of many different perspectives and skill sets. In some vague sense, quality superintelligence would be the most capable form of all, inasmuch as it could grasp and solve problems that are, for all practical purposes, beyond the
direct
reach of speed superintelligence and collective superintelligence.
¹⁴

In some domains, quantity is a poor substitute for quality. One solitary genius working out of a cork-lined bedroom can write
In Search of Lost Time
. Could an equivalent masterpiece be produced by recruiting an office building full of literary hacks?
¹⁵
Even within the range of present human variation we see that some functions benefit greatly from the labor of one brilliant mastermind as opposed to the joint efforts of myriad mediocrities. If we widen our purview to include
superintelligent
minds, we must countenance a likelihood of there being intellectual problems solvable only by superintelligence and intractable to any ever-so-large collective of non-augmented humans.

There might thus be some problems that are solvable by a quality superintelligence, and perhaps by a speed superintelligence, yet which a loosely integrated collective superintelligence cannot solve (other than by first amplifying its own intelligence).
¹⁶
We cannot clearly see what all these problems are, but we can characterize them in general terms.
¹⁷
They would tend to be problems involving multiple complex interdependencies that do not permit of independently verifiable solution steps: problems that therefore cannot be solved in a piecemeal fashion, and that might require qualitatively new kinds of understanding or new representational frameworks that are too deep or too complicated for the current edition of mortals to discover or use effectively. Some types of artistic creation and strategic cognition might fall into this category. Some types of scientific breakthrough, perhaps, likewise. And one can speculate that the tardiness and
wobbliness of humanity’s progress on many of the “eternal problems” of philosophy are due to the unsuitability of the human cortex for philosophical work. On this view, our most celebrated philosophers are like dogs walking on their hind legs—just barely attaining the threshold level of performance required for engaging in the activity
at all
.
¹⁸

Minor changes in brain volume and wiring can have major consequences, as we see when we compare the intellectual and technological achievements of humans with those of other apes. The far greater changes in computing resources and architecture that machine intelligence will enable will probably have consequences that are even more profound. It is difficult, perhaps impossible, for us to form an intuitive sense of the aptitudes of a superintelligence; but we can at least get an inkling of the space of possibilities by looking at some of the advantages open to digital minds. The hardware advantages are easiest to appreciate:

•
Speed of computational elements
. Biological neurons operate at a peak speed of about 200 Hz, a full seven orders of magnitude slower than a modern microprocessor (~ 2 GHz).
¹⁹
As a consequence, the human brain is forced to rely on massive parallelization and is incapable of rapidly performing any computation that requires a large number of sequential operations.
²⁰
(Anything the brain does in under a second cannot use much more than a hundred sequential operations—perhaps only a few dozen.) Yet many of the most practically important algorithms in programming and computer science are not easily parallelizable. Many cognitive tasks could be performed far more efficiently if the brain’s native support for parallelizable pattern-matching algorithms were complemented by, and integrated with, support for fast sequential processing.

•
Internal communication speed
. Axons carry action potentials at speeds of 120 m/s or less, whereas electronic processing cores can communicate optically at the speed of light (300,000,000 m/s).
²¹
The sluggishness of neural signals limits how big a biological brain can be while functioning as a single processing unit. For example, to achieve a round-trip latency of less than 10 ms between any two elements in a system, biological brains must be smaller than 0.11 m
³
. An electronic system, on the other hand, could be 6.1×10
¹⁷
m
³
, about the size of a dwarf planet: eighteen orders of magnitude larger.
²²

•
Number of computational elements
. The human brain has somewhat fewer than 100 billion neurons.
²³
Humans have about three and a half times the brain size of chimpanzees (though only one-fifth the brain size of sperm whales).
²⁴
The number of neurons in a biological creature is most obviously limited by cranial volume and metabolic constraints, but other factors may also be significant for larger brains (such as cooling, development time, and signal-conductance delays—see the previous point). By contrast, computer hardware is indefinitely scalable up to very high physical limits.
²⁵
Supercomputers can be warehouse-sized or larger, with additional remote capacity added via high-speed cables.
²⁶

•
Storage capacity
. Human working memory is able to hold no more than some four or five chunks of information at any given time.
²⁷
While it would be misleading to compare the size of human working memory directly with the amount of RAM in a digital computer, it is clear that the hardware advantages of digital intelligences will make it possible for them to have larger working memories. This might enable such minds to intuitively grasp complex relationships that humans can only fumblingly handle via plodding calculation.
²⁸
Human long-term memory is also limited, though it is unclear whether we manage to exhaust its storage capacity during the course of an ordinary lifetime—the rate at which we accumulate information is so slow. (On one estimate, the adult human brain stores about one billion bits—a couple of orders of magnitude less than a low-end smartphone.
²⁹
) Both the amount of information stored and the speed with which it can be accessed could thus be vastly greater in a machine brain than in a biological brain.

•
Reliability, lifespan, sensors, etc
. Machine intelligences might have various other hardware advantages. For example, biological neurons are less reliable than transistors.
³⁰
Since noisy computing necessitates redundant encoding schemes that use multiple elements to encode a single bit of information, a digital brain might derive some efficiency gains from the use of reliable high-precision computing elements. Brains become fatigued after a few hours of work and start to permanently decay after a few decades of subjective time; microprocessors are not subject to these limitations. Data flow into a machine intelligence could be increased by adding millions of sensors. Depending on the technology used, a machine might have reconfigurable hardware that can be optimized for changing task requirements, whereas much of the brain’s architecture is fixed from birth or only slowly changeable (though the details of synaptic connectivity can change over shorter timescales, like days).
³¹

At present, the computational power of the biological brain still compares favorably with that of digital computers, though top-of-the-line supercomputers are attaining levels of performance that are within the range of plausible estimates of the brain’s processing power.
³²
But hardware is rapidly improving, and the ultimate limits of hardware performance are vastly higher than those of biological computing substrates.

Digital minds will also benefit from major advantages in software:

•
Editability
. It is easier to experiment with parameter variations in software than in neural wetware. For example, with a whole brain emulation one could easily trial what happens if one adds more neurons in a particular cortical area or if one increases or decreases their excitability. Running such experiments in living biological brains would be far more difficult.

•
Duplicability
. With software, one can quickly make arbitrarily many high-fidelity copies to fill the available hardware base. Biological brains, by contrast, can be reproduced only very slowly; and each new instance starts out in a helpless state, remembering nothing of what its parents learned in their lifetimes.

•
Goal coordination
. Human collectives are replete with inefficiencies arising from the fact that it is nearly impossible to achieve complete uniformity of purpose among the members of a large group—at least until it becomes feasible to induce docility on a
large scale by means of drugs or genetic selection. A “copy clan” (a group of identical or almost identical programs sharing a common goal) would avoid such coordination problems.

•
Memory sharing
. Biological brains need extended periods of training and mentorship whereas digital minds could acquire new memories and skills by swapping data files. A population of a billion copies of an AI program could synchronize their databases periodically, so that all the instances of the program know everything that any instance learned during the previous hour. (Direct memory transfer requires standardized representational formats. Easy swapping of high-level cognitive content would therefore not be possible between just any pair of machine intelligences. In particular, it would not be possible among first-generation whole brain emulations.)

•
New modules, modalities, and algorithms
. Visual perception seems to us easy and effortless, quite unlike solving textbook geometry problems—this despite the fact that it takes a massive amount of computation to reconstruct, from the two-dimensional patterns of stimulation on our retinas, a three-dimensional representation of a world populated with recognizable objects. The reason this seems easy is that we have dedicated low-level neural machinery for processing visual information. This low-level processing occurs unconsciously and automatically, without draining our mental energy or conscious attention. Music perception, language use, social cognition, and other forms of information processing that are “natural” for us humans seem to be likewise supported by dedicated neurocomputational modules. An artificial mind that had such specialized support for other cognitive domains that have become important in the contemporary world—such as engineering, computer programming, and business strategy—would have big advantages over minds like ours that have to rely on clunky general-purpose cognition to think about such things. New algorithms may also be developed to take advantage of the distinct affordances of digital hardware, such as its support for fast serial processing.