X-Message-Number: 15919 Date: Fri, 23 Mar 2001 08:04:45 -0800 From: Lee Corbin <> Subject: Trust in All-Powerful Lords Sabine Atkins wrote >[Lee Corbin wrote] >>And so how have you planned to deal with all of us >>recalcitrants who want to live outside the control >>your AI? I guess some of us are going to have to >>band together and create a machine that is capable >>of protecting us from you. > >Yes, if the recalcitrants want to kill, steal, torture >people without their consent, etc in their everyday >lives, they should leave the solar system when the >Friendly AI has grown up. ;-) Evidently, I did not make myself clear. What _exactly_ do you intend to do with peaceful yet very advanced entities (such as many of us hope to become) who intend to live outside the control of your AI? (Please be specific.) >As far as SIAI is concerned, the super intelligent >AI we are planning to build will be a protector and >a facilitator. It is also planned to prevent us from >doing harm to each other and ourselves. Evidently you cannot see why people are finding your words alarming. Please: history is replete with the efforts of the best-intentioned people to provide "workers' paradises" and other benevolent dictatorships. Are you unaware of Lord Acton's principle? (Power corrupts and absolute power corrupts absolutely.) Do you not think that Joseph Stalin, the young revolutionary, was completely sincere in his desire to help the Russian people? Or Mao Ze Dong? My third question: by what miracle of computational science can you be sure that a tiny rogue-element has not been inserted by some programmer (or by some external fiend) into the architecture of your AI? I believe that any attempt to prove that your AI does not contain such an element is NP complete, if not much, much, harder. Fourth: So, in short, are you asking us to just "trust you, and everything will be all right"? Lee Corbin Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=15919