Read Superintelligence: Paths, Dangers, Strategies Online
Authors: Nick Bostrom
Tags: #Science, #Philosophy, #Non-Fiction
Superintelligence: Paths, Dangers, Strategies (69 page)
Tetlock, Philip E., and Belkin, Aaron. 1996. “Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives.” In
Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives
, edited by Philip E. Tetlock and Aaron Belkin, 1–38. Princeton, NJ: Princeton University Press.
Thompson, Adrian. 1997. “Artificial Evolution in the Physical World.” In
Evolutionary Robotics: From Intelligent Robots to Artificial Life
, edited by Takashi Gomi, 101–25. Er ’97. Carp, ON: Applied AI Systems.
Thrun, S., Montemerlo, M., Dahlkamp, H., Stavens, D., Aron, A., Diebel, J., Fong, P., et al. 2006. “Stanley: The Robot That Won the DARPA Grand Challenge.”
Journal of Field Robotics
23 (9): 661–92.
Trachtenberg, J. T., Chen, B. E., Knott, G. W., Feng, G., Sanes, J. R., Welker, E., and Svoboda, K. 2002. “Long-Term In Vivo Imaging of Experience-Dependent Synaptic Plasticity in Adult Cortex.”
Nature
420 (6917): 788–94.
Traub, Wesley A. 2012. “Terrestrial, Habitable-Zone Exoplanet Frequency from
Kepler
.”
Astrophysical Journal
745 (1): 1–10.
Truman, James W., Taylor, Barbara J., and Awad, Timothy A. 1993. “Formation of the Adult Nervous System.” In
The Development of
Drosophila Melanogaster, edited by Michael Bate and Alfonso Martinez Arias. Plainview, NY: Cold Spring Harbor Laboratory.
Tuomi, Ilkka. 2002. “The Lives and the Death of Moore’s Law.”
First Monday
7 (11).
Turing, A. M. 1950. “Computing Machinery and Intelligence.”
Mind
59 (236): 433–60.
Turkheimer, Eric, Haley, Andreana, Waldron, Mary, D’Onofrio, Brian, and Gottesman, Irving I. 2003. “Socioeconomic Status Modifies Heritability of IQ in Young Children.”
Psychological Science
14 (6): 623–8.
Uauy, Ricardo, and Dangour, Alan D. 2006. “Nutrition in Brain Development and Aging: Role of Essential Fatty Acids.” Supplement,
Nutrition Reviews
64 (5): S24–S33.
Ulam, Stanislaw M. 1958. “John von Neumann.”
Bulletin of the American Mathematical Society
64 (3): 1–49.
Uncertain Future, The. 2012. “Frequently Asked Questions” The Uncertain Future. Retrieved March 25, 2012. Available at
http://www.theuncertainfuture.com/faq.html
.
U.S. Congress, Office of Technology Assessment. 1995.
U.S.–Russian Cooperation in Space
ISS-618. Washington, DC: U.S. Government Printing Office, April.
Van Zanden, Jan Luiten. 2003.
On Global Economic History: A Personal View on an Agenda for Future Research
. International Institute for Social History, July 23.
Vardi, Moshe Y. 2012. “Artificial Intelligence: Past and Future.”
Communications of the ACM
55 (1): 5.
Vassar, Michael, and Freitas, Robert A., Jr. 2006. “Lifeboat Foundation Nanoshield.” Lifeboat Foundation. Retrieved May 12, 2012. Available at
http://lifeboat.com/ex/nanoshield
.
Vinge, Vernor. 1993. “The Coming Technological Singularity: How to Survive in the Post-Human Era.” In
Vision-21: Interdisciplinary Science and Engineering in the Era of Cyberspace
, 11–22. NASA Conference Publication 10129. NASA Lewis Research Center.
Visscher, P. M., Hill, W. G., and Wray, N. R. 2008. “Heritability in the Genomics Era: Concepts and Misconceptions.”
Nature Reviews Genetics
9 (4): 255–66.
Vollenweider, Franz, Gamma, Alex, Liechti, Matthias, and Huber, Theo. 1998. “Psychological and Cardiovascular Effects and Short-Term Sequelae of MDMA (‘Ecstasy’) in MDMA-Naïve Healthy Volunteers.”
Neuropsychopharmachology
19 (4): 241–51.
Wade, Michael J. 1976. “Group Selections Among Laboratory Populations of Tribolium.”
Proceedings of the National Academy of Sciences of the United States of America
73 (12): 4604–7.
Wainwright, Martin J., and Jordan, Michael I. 2008. “Graphical Models, Exponential Families, and Variational Inference.”
Foundations and Trends in Machine Learning
1 (1–2): 1–305.
Walker, Mark. 2002. “Prolegomena to Any Future Philosophy.”
Journal of Evolution and Technology
10 (1).
Walsh, Nick Paton. 2001. “Alter our DNA or robots will take over, warns Hawking.”
The Observer
, September 1.
http://www.theguardian.com/uk/2001/sep/02/medicalscience.genetics
.
Warwick, Kevin. 2002.
I, Cyborg
. London: Century.
Wehner, M., Oliker, L., and Shalf, J. 2008. “Towards Ultra-High Resolution Models of Climate and Weather.”
International Journal of High Performance Computing Applications
22 (2): 149–65.
Weizenbaum, Joseph. 1966. “Eliza: A Computer Program for the Study of Natural Language Communication Between Man And Machine.”
Communications of the ACM
9 (1): 36–45.
Weizenbaum, Joseph. 1976.
Computer Power and Human Reason: From Judgment to Calculation
. San FrancYork, CA: W. H. Freeman.
Werbos, Paul John. 1994.
The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting
. New York: Wiley.
White, J. G., Southgate, E., Thomson, J. N., and Brenner, S. 1986. “The Structure of the Nervous System of the Nematode
Caenorhabditis Elegans
.”
Philosophical Transactions of the Royal Society of London
.
Series B, Biological Sciences
314 (1165): 1–340.
Whitehead, Hal. 2003.
Sperm Whales: Social Evolution in the Ocean
. Chicago: University of Chicago Press.
Whitman, William B., Coleman, David C., and Wiebe, William J. 1998. “Prokaryotes: The Unseen Majority.”
Proceedings of the National Academy of Sciences of the United States of America
95 (12): 6578–83.
Wiener, Norbert. 1960. “Some Moral and Technical Consequences of Automation.”
Science
131 (3410): 1355–8.
Wikipedia
. 2012a, s.v. “Computer Bridge.” Retrieved June 30, 2013. Available at
http://en.wikipedia.org/wiki/Computer_bridge
.
Wikipedia
. 2012b, s.v. “Supercomputer.” Retrieved June 30, 2013. Available at
http://et.wikipedia.org/wiki/Superarvuti
.
Williams, George C. 1966.
Adaptation and Natural Selection: A Critique of Some Current Evolutionary Thought
. Princeton Science Library. Princeton, NJ: Princeton University Press.
Winograd, Terry. 1972.
Understanding Natural Language
. New York: Academic Press.
Wood, Nigel. 2007.
Chinese Glazes: Their Origins, Chemistry and Re-creation
. London: A. & C. Black.
World Bank. 2008.
Global Economic Prospects: Technology Diffusion in the Developing World
42097. Washington, DC.
World Robotics. 2011.
Executive Summary of 1
.
World Robotics 2011 Industrial Robots; 2. World Robotics 2011 Service Robots
. Retrieved June 30, 2012. Available at
http://www.bara.org.uk/pdf/2012/world-robotics/Executive_Summary_WR_2012.pdf
.
World Values Survey. 2008.
WVS 2005-2008
. Retrieved 29 October, 2013. Available at
http://www.wvsevsdb.com/wvs/WVSAnalizeStudy.jsp
.
Wright, Robert. 2001.
Nonzero: The Logic of Human Destiny
. New York: Vintage.
Yaeger, Larry. 1994. “Computational Genetics, Physiology, Metabolism, Neural Systems, Learning, Vision, and Behavior or PolyWorld: Life in a New Context.” In
Proceedings of the Artificial Life III Conference
, edited by C. G. Langton, 263–98. Santa Fe Institute Studies in the Sciences of Complexity. Reading, MA: Addison-Wesley.
Yudkowsky, Eliezer. 2001.
Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures
. Machine Intelligence Research Institute, San Francisco, CA, June 15.
Yudkowsky, Eliezer. 2002. “The AI-Box Experiment.” Retrieved January 15, 2012. Available at
http://yudkowsky.net/singularity/aibox
.
Yudkowsky, Eliezer. 2004.
Coherent Extrapolated Volition
. Machine Intelligence Research Institute, San Francisco, CA, May.
Yudkowsky, Eliezer. 2007. “Levels of Organization in General Intelligence.” In
Artificial General Intelligence
, edited by Ben Goertzel and Cassio Pennachin, 389–501. Cognitive Technologies. Berlin: Springer.
Yudkowsky, Eliezer. 2008a. “Artificial Intelligence as a Positive and Negative Factor in Global Risk.” In
Global Catastrophic Risks
, edited by Nick Bostrom and Milan M. Ćirković, 308–45. New York: Oxford University Press.
Yudkowsky, Eliezer. 2008b. “Sustained Strong Recursion.”
Less Wrong
(blog), December 5.
Yudkowsky, Eliezer. 2010.
Timeless Decision Theory
. Machine Intelligence Research Institute, San Francisco, CA.
Yudkowsky, Eliezer. 2011.
Complex Value Systems are Required to Realize Valuable Futures
. San Francisco, CA: Machine Intelligence Research Institute.
Yudkowsky, Eliezer. 2013.
Intelligence Explosion Microeconomics
, Technical Report 2013–1. Berkeley, CA: Machine Intelligence Research Institute.
Zahavi, Amotz, and Zahavi, Avishag. 1997.
The Handicap Principle: A Missing Piece of Darwin’s Puzzle
. Translated by N. Zahavi-Ely and M. P. Ely. New York: Oxford University Press.
Zalasiewicz, J., Williams, M., Smith, A., Barry, T. L., Coe, A. L., Bown, P. R., Brenchley, P., et al. 2008. “Are We Now Living in the Anthropocene?”
GSA Today
18 (2): 4–8.
Zeira, Joseph. 2011. “Innovations, Patent Races and Endogenous Growth.”
Journal of Economic Growth
16 (2): 135–56.
Zuleta, Hernando. 2008. “An Empirical Note on Factor Shares.”
Journal of International Trade and Economic Development
17 (3): 379–90.
A
Afghan Taliban
215
Agricultural Revolution
2
,
80
,
261
AI-complete problem
14
,
47
,
71
,
93
,
145
,
186
AI-OUM,
see
optimality notions
AI-RL,
see
optimality notions
AI-VL,
see
optimality notions
algorithmic soup
172
anthropics
27
–
28
,
126
,
134
–
135
,
174
,
222
–
225
definition
225
Arendt, Hannah
105
Armstrong, Stuart
280
,
291
,
294
,
302
artificial agent
10
,
88
,
105
–
109
,
172
–
176
,
185
–
206
;
see also
Bayesian agent
artificial intelligence
greater-than-human,
see
superintelligence
overprediction of
4
Asimov, Isaac
139
autism
57
automata theory
5
automatic circuit breaker
17
B
backgammon
12
backpropagation algorithm
8
bargaining costs
182
Bayesian agent
9
–
11
,
123
,
130
;
see also
artificial agent
and
optimality notions
Bayesian networks
9
Berliner, Hans
12
biological cognition
22
,
36
–
48
,
50
–
51
,
232
biological enhancement
36
–
48
,
50
–
51
,
142
–
143
,
232
;
see also
cognitive enhancement
informational
130
brain implant,
see
cyborg
brain plasticity
48
brain–computer interfaces
44
–
48
,
51
,
83
,
142
–
143
;
see also
cyborg
Brown, Louise
43
C
capability control
129
–
144
,
156
–
157
capital
39
,
48
,
68
,
84
–
88
,
99
,
113
–
114
,
159
–
184
,
251
,
287
,
288
,
289
causal validity semantics
197
CEV,
see
coherent extrapolated volition
Chalmers, David
24
,
265
,
283
,
295
,
302
character recognition
15
checkers
12
child machine
23
,
29
;
see also
seed AI
CHINOOK
12
civilization baseline
63
cloning
42
cognitive enhancement
42
–
51
,
67
,
94
,
111
–
112
,
193
,
204
,
232
–
238
,
244
,
259
coherent extrapolated volition (CEV)
198
,
211
–
227
,
296
,
298
,
303
definition
211
collaboration (benefits of)
249
collective intelligence
48
–
51
,
52
–
57
,
67
,
72
,
142
,
163
,
203
,
259
,
271
,
273
,
279
collective superintelligence
39
,
48
–
49
,
52
–
59
,
83
,
93
,
99
,
285
definition
54
combinatorial explosion
6
,
9
,
10
,
47
,
155
common sense
14
computer vision
9
computing power
7
–
9
,
24
,
25
–
35
,
47
,
53
–
60
,
68
–
77
,
101
,
134
,
155
,
198
,
240
–
244
,
251
,
286
,
288
;
see also
computronium
and
hardware overhang
computronium
101
,
123
–
124
,
140
,
193
,
219
;
see also
computing power
connectionism
8
consciousness
22
,
106
,
126
,
139
,
173
–
176
,
216
,
226
,
271
,
282
,
288
,
292
,
303
;
see also
mind crime
control methods
127
–
144
,
145
–
158
,
202
,
236
–
238
,
286
;
see also
capability control
and
motivation selection
Copernicus, Nicolaus
14
cosmic endowment
101
–
104
,
115
,
134
,
209
,
214
–
217
,
227
,
250
,
260
,
283
,
296
crosswords (solving)
12
cryptographic reward tokens
134
,
276
cryptography
80
D
DARPA,
see
Defense Advanced Research Projects Agency
DART (tool)
15
Dartmouth Summer Project
5
decision support systems
15
,
98
;
see also
tool-AI
decision theory
10
–
11
,
88
,
185
–
186
,
221
–
227
,
280
,
298
;
see also
optimality notions
decisive strategic advantage
78
–
89
,
95
,
104
–
112
,
115
–
126
,
129
–
138
,
148
–
149
,
156
–
159
,
177
,
190
,
209
–
214
,
225
,
252
Deep Blue
12
Deep Fritz
22
Defense Advanced Research Projects Agency (DARPA)
15
design effort,
see
optimization power
Dewey, Daniel
291
Differential Technological Development (Principle of)
230
–
237
Diffie–Hellman key exchange protocol
80
diminishing returns
37
–
38
,
66
,
88
,
114
,
273
,
303
direct reach
58
domesticity
140
–
143
,
146
–
156
,
187
,
191
,
207
,
222
Drexler, Eric
239
,
270
,
276
,
278
,
300
Dutch book
111
E
economic growth
3
,
160
–
166
,
179
,
261
,
274
,
299
ELIZA (program)
6
emulation modulation
207
Enigma code
87
environment of evolutionary adaptedness
164
,
171
equation solvers
15
Eurisko
12
evolution
8
–
9
,
23
–
27
,
44
,
154
,
173
–
176
,
187
,
198
,
207
,
265
,
266
,
267
,
273
evolutionary selection
187
,
207
,
290
evolvable hardware
154
exhaustive search
6
existential risk
4
,
21
,
55
,
100
–
104
,
115
–
126
,
175
,
183
,
230
–
236
,
239
–
254
,
256
–
259
,
286
,
301
–
302
step risks
233
expert system
7
explicit representation
207
exponential growth,
see
growth
external reference semantics
197
F
face recognition
15
Faraday cage
130
Fifth-Generation Computer Systems Project
7
fitness function
25
;
see also
evolution
FreeCell (game)
13
G
General Problem Solver
6
genetic algorithms
7
–
13
,
24
–
27
,
237
–
240
;
see also
evolution
genetic selection
37
–
50
,
61
,
232
–
238
;
see also
evolution
definition
148
genotyping
37
germline interventions
37
–
44
,
67
,
273
;
see also
embryo selection
Ginsberg, Matt
12
Go (game)
13
goal-content
109
–
110
,
146
,
207
,
222
–
227
Good Old-Fashioned Artificial Intelligence (GOFAI)
7
–
15
,
23
Good, I. J.
4
Gorbachev, Mikhail
86
graceful degradation
7
graphical models
11
growth
1
–
7
,
48
–
55
,
69
–
75
,
83
,
163
,
261
,
281
H
