Superintelligence: Paths, Dangers, Strategies (51 page)

24
. Sandberg (2013).

25
. See the “Computer requirements” section of Sandberg and Bostrom (2008, 79–81).

26
. A lower level of success might be a brain simulation that has biologically suggestive micro-dynamics and displays a substantial range of emergent species-typical activity such as a slow-wave sleep state or activity-dependent plasticity. Whereas such a simulation could be a useful testbed for neuroscientific research (though one which might come close to raising serious ethical issues), it would not count as a whole brain emulation unless the simulation were sufficiently accurate to be able to perform a substantial fraction of the intellectual work that the simulated brain was capable of. As a rule of thumb, we might say that in order for a simulation of a human brain to count as a whole brain emulation, it would need to be able to express coherent verbal thoughts or have the capacity to learn to do so.

27
. Sandberg and Bostrom (2008).

28
. Sandberg and Bostrom (2008). Further explanation can be found in the original report.

29
. The first map is described in Albertson and Thomson (1976) and White et al. (1986). The combined (and in some cases corrected) network is available from the “WormAtlas” website (
http://www.wormatlas.org/
).

30
. For a review of past attempts of emulating
C. elegans
and their fates, see Kaufman (2011). Kaufman quotes one ambitious doctoral student working in the area, David Dalrymple, as saying,
“With optogenetic techniques, we are just at the point where it’s not an outrageous proposal to reach for the capability to read and write to anywhere in a living
C. elegans
nervous system, using a high-throughput automated system…. I expect to be finished with
C. elegans
in 2–3 years. I would be extremely surprised, for whatever that’s worth, if this is still an open problem in 2020” (Dalrymple 2011). Brain models aiming for biological realism that were hand-coded (rather than generated automatically) have achieved some basic functionality; see, e.g., Eliasmith et al. (2012).

31
.
Caenorhabditis elegans
does have some convenient special properties. For example, the organism is transparent, and the wiring pattern of its nervous system does not change between individuals.

32
. If neuromorphic AI rather than whole brain emulation is the end product, then it might or might not be the case that the relevant insights would be derived through attempts to simulate
human
brains. It is conceivable that the important cortical tricks would be discovered during the study of (nonhuman) animal brains. Some animal brains might be easier to work with than human brains, and smaller brains would require fewer resources to scan and model. Research on animal brains would also be subject to less regulation. It is even conceivable that the first human-level machine intelligence will be created by completing a whole brain emulation of some suitable animal and then finding ways to enhance the resultant digital mind. Thus humanity could get its comeuppance from an uplifted lab mouse or macaque.

33
. Uauy and Dangour (2006); Georgieff (2007); Stewart et al. (2008); Eppig et al. (2010); Cotman and Berchtold (2002).

34
. According to the World Health Organization in 2007, nearly 2 billion individuals have insufficient iodine intake (
The Lancet
2008).
Severe
iodine deficiency hinders neurological development and leads to cretinism, which involves an average loss of about 12.5 IQ points (Qian et al. 2005). The condition can be easily and inexpensively prevented though salt fortification (Horton et al. 2008).

35
. Bostrom and Sandberg (2009a).

36
. Bostrom and Sandberg (2009b). A typical
putative
performance increase from pharmacological and nutritional enhancement is in the range of 10–20% on test tasks measuring working memory, attention, etc. But it is generally dubious whether such reported gains are real, sustainable over a longer term, and indicative of correspondingly improved results in real-world problem situations (Repantis et al. 2010). For instance, in some cases there might be a compensating deterioration on some performance dimensions that are not measured by the test tasks (Sandberg and Bostrom 2006).

37
. If there were an easy way to enhance cognition, one would expect evolution already to have taken advantage of it. Consequently, the most promising kind of nootropic to investigate may be one that promises to boost intelligence in some manner that we can see would have lowered fitness in the ancestral environment—for example, by increasing head size at birth or amping up the brain’s glucose metabolism. For a more detailed discussion of this idea (along with several important qualifications), see Bostrom (2009b).

38
. Sperm are harder to screen because, in contrast to embryos, they consist of only one cell—and one cell needs to be destroyed in order to do the sequencing. Oocytes also consist of only one cell; however, the first and second cell divisions are asymmetric and produce one daughter cell with very little cytoplasm, the polar body. Since polar bodies contain the same genome as the main cell and are redundant (they eventually degenerate) they can be biopsied and used for screening (Gianaroli 2000).

39
. Each of these practices was subject to some ethical controversy when it was introduced, but there seems to be a trend toward increasing acceptance. Attitudes toward human genetic engineering and embryo selection vary significantly across cultures, suggesting that development and application of new techniques will probably take place even if some countries initially adopt a cautious stance, although the rate at which this happens will be influenced by moral, religious, and political pressures.

40
. Davies et al. (2011); Benyamin et al. (2013); Plomin et al. (2013). See also Mardis (2011); Hsu (2012).

41
.
Broad-sense heritability of adult IQ is usually estimated in the range of 0.5–0.8 within middle-class strata of developed nations (Bouchard 2004, 148). Narrow-sense heritability, which measures the portion of variance that is attributable to additive genetic factors, is lower (in the range 0.3–0.5) but still substantial (Devlin et al. 1997; Davies et al. 2011; Visscher et al. 2008). These estimates could change for different populations and environments, as heritabilities vary depending on the population and environment being studied. For example, lower heritabilities have been found among children and those from deprived environments (Benyamin et al. 2013; Turkheimer et al. 2003). Nisbett et al. (2012) review numerous environmental influences on variation in cognitive ability.

42
. The following several paragraphs draw heavily on joint work with Carl Shulman (Shulman and Bostrom 2014).

43
. This table is taken from Shulman and Bostrom (2014). It is based on a toy model that assumes a Gaussian distribution of predicted IQs among the embryos with a standard deviation of 7.5 points. The amount of cognitive enhancement that would be delivered with different numbers of embryos depends on how different the embryos are from one another in the additive genetic variants whose effects we know. Siblings have a coefficient of relatedness of ½, and common additive genetic variants account for half or less of variance in adult fluid intelligence (Davies et al. 2011). These two facts suggest that where the observed population standard deviation in developed countries is 15 points, the standard deviation of genetic influences within a batch of embryos would be 7.5 points or less.

44
. With imperfect information about the additive genetic effects on cognitive ability, effect sizes would be reduced. However, even a small amount of knowledge would go a relatively long way, because the gains from selection do not scale linearly with the portion of variance that we can predict. Instead, the effectiveness of our selection depends on the standard deviation of predicted mean IQ, which scales as the
square root
of variance. For example, if one could account for 12.5% of the variance, this could deliver effects half as great as those in
Table 1
, which assume 50%. For comparison, a recent study (Rietveld et al. 2013) claims to have already identified 2.5% of the variance.

45
. For comparison, standard practice today involves the creation of fewer than ten embryos.

46
. Adult and embryonic stem cells can be coaxed to develop into sperm cells and oocytes, which can then be fused to produce an embryo (Nagy et al. 2008; Nagy and Chang 2007). Egg cell precursors can also form parthenogenetic blastocysts, unfertilized and non-viable embryos, able to produce embryonic stem cell lines for the process (Mai et al. 2007).

47
. The opinion is that of Katsuhiko Hayashi, as reported in Cyranoski (2013). The Hinxton Group, an international consortium of scientists that discusses stem cell ethics and challenges, predicted in 2008 that human stem cell-derived gametes would be available within ten years (Hinxton Group 2008), and developments thus far are broadly consistent with this.

48
. Sparrow (2013); Miller (2012); The Uncertain Future (2012).

49
. Sparrow (2013).

50
. Secular concerns might focus on anticipated impacts on social inequality, the medical safety of the procedure, fears of an enhancement “rat race,” rights and responsibilities of parents vis-à-vis their prospective offspring, the shadow of twentieth-century eugenics, the concept of human dignity, and the proper limits of states’ involvement in the reproductive choices of their citizens. (For a discussion of the ethics of cognitive enhancement see Bostrom and Ord [2006], Bostrom and Roache [2011], and Sandberg and Savulescu [2011].) Some religious traditions may offer additional concerns, including ones centering on the moral status of embryos or the proper limits of human agency within the scheme of creation.

51
. To stave off the negative effects of inbreeding, iterated embryo selection would require either a large starting supply of donors or the expenditure of substantial selective power to reduce harmful recessive alleles. Either alternative would tend to push toward offspring being less closely genetically related to their parents (and more related to one another).

52
. Adapted from Shulman and Bostrom (2014).

53
. Bostrom (2008b).

54
. Just how difficult an obstacle epigenetics will be is not yet known (Chason et al. 2011; Iliadou et al. 2011).

55
.
While cognitive ability is a fairly heritable trait, there may be few or no
common
alleles or polymorphisms that individually have a large positive effect on intelligence (Davis et al. 2010; Davies et al. 2011; Rietveld et al. 2013). As sequencing methods improve, the mapping out of low-frequency alleles and their cognitive and behavioral correlates will become increasingly feasible. There is some theoretical evidence suggesting that some alleles that cause genetic disorders in homozygotes may provide sizeable cognitive advantages in heterozygote carriers, leading to a prediction that Gaucher, Tay-Sachs, and Niemann-Pick heterozygotes would be about 5 IQ points higher than control groups (Cochran et al. 2006). Time will tell whether this holds.

56
. One paper (Nachman and Crowell 2000) estimates 175 mutations per genome per generation. Another (Lynch 2010), using different methods, estimates that the average newborn has between 50 and 100 new mutations, and Kong et al. (2012) implies a figure of around 77 new mutations per generation. Most of these mutations do not affect functioning, or do so only to an imperceptibly slight degree; but the combined effects of many very slightly deleterious mutations could be a significant loss of fitness. See also Crow (2000).

57
. Crow (2000); Lynch (2010).

58
. There are some potentially important caveats to this idea. It is possible that the modal genome would need some adjustments in order to avoid problems. For example, parts of the genome might be adapted to interacting with other parts under the assumption that all parts function with a certain level of efficiency. Increasing the efficiency of those parts might then lead to overshooting along some metabolic pathways.

59
. These composites were created by Mike Mike from individual photographs taken by Virtual Flavius (Mike 2013).

60
. They can, of course, have some effects sooner—for instance, by changing people’s expectations of what is to come.

61
. Louis Harris & Associates (1969); Mason (2003).

62
. Kalfoglou et al. (2004).

63
. The data is obviously limited, but individuals selected for 1-in-10,000 results on childhood ability tests have been shown, in longitudinal studies, to be substantially more likely to become tenured professors, earn patents, and succeed in business than those with slightly less exceptional scores (Kell et al. 2013). Roe (1953) studied sixty-four eminent scientists and found median cognitive ability three to four standard deviations above the population norm and strikingly higher than is typical for scientists in general. (Cognitive ability is also correlated with lifetime earnings and with non-financial outcomes such as life expectancy, divorce rates, and probability of dropping out of school [Deary 2012].) An upward shift of the distribution of cognitive ability would have disproportionately large effects at the tails, especially increasing the number of highly gifted and reducing the number of people with retardation and learning disabilities. See also Bostrom and Ord (2006) and Sandberg and Savulescu (2011).