[D66] AI: The Artilect War (2)

[René Oudeweg]

https://memeinnovation.wordpress.com/2020/06/16/notes-on-the-artilect-war-cosmists-vs-terrans-by-hugo-de-garis/

Notes on The Artilect War: Cosmists vs Terrans by Hugo de Garis

Civilization advances by extending the number of important operations 
which we can perform without thinking of them. — Alfred North Whitehead

It is difficult to get a man to understand something, when his salary 
depends upon his not understanding it. — Upton Sinclair

Let an ultraintelligent machine be defined as a machine that can far 
surpass all the intellectual activities of any man however clever. Since 
the design of machines is one of these intellectual activities, an 
ultraintelligent machine could design even better machines; there would 
then unquestionably be an ‘intelligence explosion’, and the intelligence 
of man would be left far behind. Thus the first ultraintelligent machine 
is the last invention that man need ever make, provided that the machine 
is docile enough to tell us how to keep it under control. — I. J. Good

nanobots will … go into the brain via capillaries and connect our 
neocortex (the outer layer of the brain where we do our thinking) to the 
cloud. So today, just as we can access many thousands of computers in 
the cloud when we need them, in the 2030s and beyond we will be able to 
access additional neocortex to think deeper thoughts. — Ray Kurzweil in 
the forward to Virtually Human (p. xi)

In his book The Artilect War (2005), Hugo de Garis envisions a world 
where human-kind responds to the rise of artilects – i.e. artificial 
intellects – by dividing into two groups: Terrans and Cosmists.

However, as I see it, there are really four camps:

1) Doubting Thomas Buffs – those who think that the technology needed to 
create artilects (i.e. beings with human-level artificial intelligence) 
is either impossible or very far off in the future;

2) Terrans – those who want to destroy artilects if they exist – or, if 
they do not exist, to prevent them from being developed in the first place;

3) Control Buff Cosmists – those who want to continue developing 
artilects, but only as brainwashed slaves that are “humanity friendly”;

4) Transhumanist Cosmists – those who want to merge with artilects – 
i.e. to become transhuman cyborgs – so as to control AI technology [1].

Note that people may transition from one camp to another. For example, 
if there are convincing “sparks of AGI,” then some Doubting Thomas Buffs 
may become Terrans.

Also, when AGIs first start to emerge, there will be many Control Buff 
Cosmists, which makes sense to me since we will still know so little 
about the “baby” AGIs that emerge, and there will be a vast number of 
possible forms of AGI.

My guess is that over time more and more people in the Control Buff camp 
will slowly transition to the Transhumanist camp.

When a given person decides to join the transhumanist camp, it may take 
a long time for that person’s body and brain to make the transition. In 
fact, society may insist that the physical/mental transition has to 
proceed in very small increments.

In my view, society will always want to keep the world safe for those 
biological humans who want to remain the same.

Elon Musk and Ray Kurzweil are both Transhumanist Cosmists. However, 
Kurzweil takes this position because he thinks that merging with AI 
technology will ultimately be a big plus (on balance) for humanity.

Musk, on the other hand, funded the development of brain-computer 
interface technology for defensive reasons. He fears that a 
superintelligence will take over if we don’t merge with technology; he 
doubts that mere unenhanced humans will be able to control the rise of a 
SuperAI.

Stuart Russell, the well known AI textbook author, would fall into the 
Control Buff Cosmist camp. He believes we may be able to pull off a 
controlled intelligence explosion. However, we would first need to 
figure out a way to safely put purpose into the machine. To do that, he 
wants the AI research community to pay much more attention to something 
called Inverse Reinforcement Learning. His latest AI textbook make IRL a 
central focus. IRL involves creating AI software that tries to figure 
out what humanity really wants. Russell seems to think that this problem 
is too difficult for humans to solve without the aid of AI. He uses the 
analogy of the King Midas problem in combination with the story of the 
magic genie that grants three wishes – i.e. how do you ask the magic 
genie to grant your wishes in a safe way. In the story, the genie’s 
owner always messes up the first wish and ends up using his last two 
wishes desperately trying to undo the problems caused by the first wish.

Hugo de Garis doesn’t spend time discussing camp (a) in his book The 
Artilect War, but he does discuss the other three camps. The Terrans 
represent camp (b). Cosmists, on the other hand, are split between camp 
(c) and (d).

Here are some examples of where AI authors would fit in this scheme:

Bill Joy would probably join camp (b),
Nick Bostrom would join camp (c), and
Hugo de Garis would also join camp (d).

It’s interesting to note that Nick Bostrom is now calling for a Global 
Surveillance State [2], in part to keep tabs on AI researchers [3]. It 
would be interesting to find out if Bill Joy supports this idea.

Footnotes

1) The Brain-Computer Interface as Educational Technology
https://innovationmemes.blogspot.com/2019/07/the-brain-computer-interface-as.html

2) Reading List for the Digital Surveillance State, Surveillance 
Capitalism, the Persuasion Industry, and the Rise of SuperAI
https://innovationmemes.blogspot.com/2019/07/reading-list-for-digital-surveillance.html

3) An Oxford philosopher who’s inspired Elon Musk thinks mass 
surveillance might be the only way to save humanity from doom
by Aria Bendix
Apr. 19, 2019

https://www.businessinsider.com/nick-bostrom-mass-surveillance-could-save-humanity-2019-4

“””
At the TED 2019 conference in Vancouver, Canada, [Nick] Bostrom posited 
another idea: that humanity could destroy itself with a technology of 
our own creation.

Bostrom said one of the only ways to save ourselves from this doomsday 
scenario would be to institute global mass surveillance.

The idea is especially controversial given the conference’s focus on 
privacy in the digital era.
“””

Wikipedia Reference on Hugo de Garis

Hugo de Garis
https://en.wikipedia.org/wiki/Hugo_de_Garis

“””
Hugo de Garis (born 1947, Sydney, Australia) is a retired researcher in 
the sub-field of artificial intelligence (AI) known as evolvable 
hardware. He became known in the 1990s for his research on the use of 
genetic algorithms to evolve artificial neural networks using 
three-dimensional cellular automata inside field programmable gate 
arrays. He claimed that this approach would enable the creation of what 
he terms “artificial brains” which would quickly surpass human levels of 
intelligence.

…

In 2005, de Garis published a book describing his views on this topic 
entitled The Artilect War: Cosmists vs. Terrans: A Bitter Controversy 
Concerning Whether Humanity Should Build Godlike Massively Intelligent 
Machines.
“””

Cosmists

“””
Cosmism is a moral philosophy that favours building or growing strong 
artificial intelligence and ultimately leaving Earth to the Terrans, who 
oppose this path for humanity. The first half of the book describes 
technologies which he believes will make it possible for computers to be 
billions or trillions of times more intelligent than humans. He predicts 
that as artificial intelligence improves and becomes progressively more 
human-like, differing views will begin to emerge regarding how far such 
research should be allowed to proceed. Cosmists will foresee the 
massive, truly astronomical potential of substrate-independent 
cognition, and will therefore advocate unlimited growth in the 
designated fields, in the hopes that “super intelligent” machines might 
one day colonise the universe. It is this “cosmic” view of history, in 
which the fate of one single species, on one single planet, is seen as 
insignificant next to the fate of the known universe, that gives the 
Cosmists their name.

Hugo identifies with that group and noted that it “would be a cosmic 
tragedy if humanity freezes evolution at the puny human level”.
“””

Terrans

“””
Terrans, on the other hand, will have a more “terrestrial” Earth-centred 
view, in which the fate of the Earth and its species (like humanity) are 
seen as being all-important. To Terrans, a future without humans is to 
be avoided at all costs, as it would represent the worst-case scenario. 
As such, Terrans will find themselves unable to ignore the possibility 
that super intelligent machines might one day cause the destruction of 
the human race—being very immensely intelligent and so cosmically 
inclined, these artilect machines may have no more moral or ethical 
difficulty in exterminating humanity than humans do in using medicines 
to cure diseases. So, Terrans will see themselves as living during the 
closing of a window of opportunity, to disable future artilects before 
they are built, after which humans will no longer have a say in the 
affairs of intelligent machines.
“””

Dystopian vision of a world at war

“””
It is these two extreme ideologies which de Garis believes may herald a 
new world war, wherein one group with a “grand plan” (the Cosmists) will 
be rabidly opposed by another which feels itself to be under deadly 
threat from that plan (the Terrans). The factions, he predicts, may 
eventually war to the death because of this, as the Terrans will come to 
view the Cosmists as “arch-monsters” when they begin seriously 
discussing acceptable risks, and the probabilities of large percentages 
of Earth-based life going extinct. In response to this, the Cosmists 
will come to view the Terrans as being reactionary extremists, and will 
stop treating them and their ideas seriously, further aggravating the 
situation, possibly beyond reconciliation.

Throughout his book, de Garis states that he is ambivalent about which 
viewpoint he ultimately supports, and attempts to make convincing cases 
for both sides. He elaborates towards the end of the book that the more 
he thinks about it, the more he feels like a Cosmist, because he feels 
that despite the horrible possibility that humanity might ultimately be 
destroyed, perhaps inadvertently or at least indifferently, by the 
artilects, he cannot ignore the fact that the human species is just 
another link in the evolutionary chain, and must become extinct in their 
current form anyway, whereas the artilects could very well be the next 
link in that chain and therefore would be excellent candidates to carry 
the torch of science and exploration forward into the rest of the universe.
“””

A third group: cyborgs

“””
De Garis also predicts a third group that will emerge between the two. 
He refers to this third party as Cyborgians or Cyborgs, because they 
will not be opposed to artilects as such, but desire to become artilects 
themselves by adding components to their own human brains, rather than 
falling into obsolescence. They will seek to become artilects by 
gradually merging themselves with machines and think that the dichotomy 
between the Cosmists and Terrans can be avoided because all human beings 
would become artilects.

[Those in the] transhumanist movement are usually identified as Cyborgians.

De Garis’ concept of the Cyborgians might have stemmed from a 
conversation with Kevin Warwick: in 2000, de Garis noted, “Just out of 
curiosity, I asked Kevin Warwick whether he was a Terran or a Cosmist. 
He said he was against the idea of artilects being built (i.e., he is 
Terran). I was surprised, and felt a shiver go up my spine. That moment 
reminded me of a biography of Lenin that I had read in my 20s in which 
the Bolsheviks and the Mensheviks first started debating the future 
government of Russia. What began as an intellectual difference ended up 
as a Russian civil war after 1917 between the white and the red Russians”.
“””

Other Wikipedia References

Bostrom, Nick
https://en.wikipedia.org/wiki/Nick_Bostrom

“””
Nick Bostrom (/ˈbɒstrəm/; Swedish: Niklas Boström [²buːstrœm]; born 10 
March 1973) is a Swedish philosopher at the University of Oxford known 
for his work on existential risk, the anthropic principle, human 
enhancement ethics, superintelligence risks, and the reversal test. In 
2011, he founded the Oxford Martin Programme on the Impacts of Future 
Technology, and is the founding director of the Future of Humanity 
Institute at Oxford University.

Bostrom is the author of over 200 publications, including 
Superintelligence: Paths, Dangers, Strategies (2014), a New York Times 
bestseller and Anthropic Bias: Observation Selection Effects in Science 
and Philosophy (2002). In 2009 and 2015, he was included in Foreign 
Policy’s Top 100 Global Thinkers list. Bostrom believes there are 
potentially great benefits from Artificial General Intelligence, but 
warns it might very quickly transform into a superintelligence that 
would deliberately extinguish humanity out of precautionary 
self-preservation or some unfathomable motive, making solving the 
problems of control beforehand an absolute priority. His book on 
superintelligence was recommended by both Elon Musk and Bill Gates. 
However, Bostrom has expressed frustration that the reaction to its 
thesis typically falls into two camps, one calling his recommendations 
absurdly alarmist because creation of superintelligence is unfeasible, 
and the other deeming them futile because superintelligence would be 
uncontrollable. Bostrom notes that both these lines of reasoning 
converge on inaction rather than trying to solve the control problem 
while there may still be time.
“””

Joy, Bill
https://en.wikipedia.org/wiki/Bill_Joy

“””
William Nelson Joy (born November 8, 1954) is an American computer 
engineer. He co-founded Sun Microsystems in 1982 along with Scott 
McNealy, Vinod Khosla, and Andy Bechtolsheim, and served as chief 
scientist at the company until 2003. He played an integral role in the 
early development of BSD UNIX while a graduate student at Berkeley, and 
he is the original author of the vi text editor. He also wrote the 2000 
essay Why The Future Doesn’t Need Us, in which he expressed deep 
concerns over the development of modern technologies.
“””

Why the future doesn’t need us.
http://longnow.org/media/djlongnow_media/press/pdf/0200004-Joy-Whythefuturedoesntneedus.pdf

Kurzweil, Ray
https://en.wikipedia.org/wiki/Ray_Kurzweil

“””
Raymond Kurzweil (/ˈkɜːrzwaɪl/ KURZ-wyle; born February 12, 1948) is an 
American inventor and futurist. He is involved in fields such as optical 
character recognition (OCR), text-to-speech synthesis, speech 
recognition technology, and electronic keyboard instruments. He has 
written books on health, artificial intelligence (AI), transhumanism, 
the technological singularity, and futurism. Kurzweil is a public 
advocate for the futurist and transhumanist movements, and gives public 
talks to share his optimistic outlook on life extension technologies and 
the future of nanotechnology, robotics, and biotechnology.

Kurzweil received the 1999 National Medal of Technology and Innovation, 
the United States’ highest honor in technology, from President Clinton 
in a White House ceremony. He was the recipient of the $500,000 
Lemelson-MIT Prize for 2001. And in 2002 he was inducted into the 
National Inventors Hall of Fame, established by the U.S. Patent Office. 
He has received 21 honorary doctorates, and honors from three U.S. 
presidents. The Public Broadcasting Service (PBS) included Kurzweil as 
one of 16 “revolutionaries who made America” along with other inventors 
of the past two centuries. Inc. magazine ranked him #8 among the “most 
fascinating” entrepreneurs in the United States and called him “Edison’s 
rightful heir”.

Kurzweil has written seven books, five of which have been national 
bestsellers. The Age of Spiritual Machines has been translated into 9 
languages and was the #1 best-selling book on Amazon in science. 
Kurzweil’s book The Singularity Is Near was a New York Times bestseller, 
and has been the #1 book on Amazon in both science and philosophy. 
Kurzweil speaks widely to audiences both public and private and 
regularly delivers keynote speeches at industry conferences like DEMO, 
SXSW, and TED. He maintains the news website KurzweilAI.net, which has 
over three million readers annually.

Kurzweil has been employed by Google since 2012, where he is a “director 
of engineering”.
“””