X-Message-Number: 15919
Date: Fri, 23 Mar 2001 08:04:45 -0800
From: Lee Corbin <>
Subject: Trust in All-Powerful Lords

Sabine Atkins wrote
>[Lee Corbin wrote]
>>And so how have you planned to deal with all of us
>>recalcitrants who want to live outside the control
>>your AI?   I guess some of us are going to have to
>>band together and create a machine that is capable
>>of protecting us from you.
>
>Yes, if the recalcitrants want to kill, steal, torture
>people without their consent, etc in their everyday
>lives, they should leave the solar system when the
>Friendly AI has grown up. ;-) 

Evidently, I did not make myself clear.  What _exactly_
do you intend to do with peaceful yet very advanced
entities (such as many of us hope to become) who intend
to live outside the control of your AI?  (Please be
specific.)

>As far as SIAI is concerned, the super intelligent
>AI we are planning to build will be a protector and
>a facilitator. It is also planned to prevent us from
>doing harm to each other and ourselves. 

Evidently you cannot see why people are finding your
words alarming.  Please: history is replete with the
efforts of the best-intentioned people to provide 
"workers' paradises" and other benevolent dictatorships.

Are you unaware of Lord Acton's principle?  (Power
corrupts and absolute power corrupts absolutely.)
Do you not think that Joseph Stalin, the young 
revolutionary, was completely sincere in his desire
to help the Russian people?  Or Mao Ze Dong?

My third question:  by what miracle of computational
science can you be sure that a tiny rogue-element has
not been inserted by some programmer (or by some 
external fiend) into the architecture of your AI?
I believe that any attempt to prove that your AI does
not contain such an element is NP complete, if not 
much, much, harder.

Fourth:  So, in short, are you asking us to just
"trust you, and everything will be all right"?

Lee Corbin

Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=15919