Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!bcm!dimacs.rutgers.edu!aramis.rutgers.edu!athos.rutgers.edu!nanotech From: cognos!geovision!gd@dciem (Gord Deinstadt) Newsgroups: sci.nanotech Subject: Re: Some problems of super-intelligence Message-ID: Date: 9 Jan 91 22:27:39 GMT Sender: nanotech@athos.rutgers.edu Organization: GeoVision Corp., Ottawa, Ontario Lines: 83 Approved: nanotech@aramis.rutgers.edu dmocsny@minerva.che.uc.edu (Daniel Mocsny) writes: >[I wrote] >>However, I wouldn't risk a network connection to my brain. Think of the >>danger of viruses; whole populations gone mad. Or docile. Or having >>their memories modified. >Think of the danger of biological viruses that exist today. Do you >live in a hermetically sealed bubble? My brain does. Ever heard of the blood-brain barrier? Of course it's not a hermetic seal, it's a selective membrane, but it is designed to physically screen out viruses and other nasties. They're too big to get through the pores. Recent AIDS research reveals that it is imperfect, however. :( >Defending against computer viruses is much easier than defending >against biological viruses. Biological viruses have many avenues of >entry into your body, and you are unable to close all of them. >Computer viruses have only one avenue of entry: you download a >set of instructions and run them. We are talking about a neural network here. In a neural network there is no distinction between data and program. Also, in the context which you trimmed out, the downloaded data was to appear to me as my own thought. If the thought is "politician X is God" and I believe that this revelation about Him came from within me, then the damage has been done. I don't know if this qualifies as a virus, but inasmuch as it profoundly alters my subsequent behaviour it certainly acts like one. It needn't be anything so obvious, anyway. Just edit the history books and supply everyone with new "facts". If done gradually probably noone would notice. Tyrants try to do this all the time, and to some extent they succeed even though they don't have direct access to the brain. Even sincere people do it, convincing other people and themselves that it didn't really happen that way... if your memories are in someone else's care, you are no longer free. > Also, when a biological virus >appears, there is nobody we can potentially locate and throw in jail. Which might deter the small fry, but the danger is the big fish, like politician X. >Therefore, you will allocate some of your computer resources to a >logical immune system, just as you today allocate some of your >biological resources to a biological immune system. Can you suggest a mechanism, in principle, for such a thing? The only thing I can think of is to somehow label data as coming from "out there". Perhaps the only safe connection is via the senses, ie. you hear the computer's voice. But this severely limits what it can do for you. > Here is one major >advantage your logical immune will have: it can learn from the >experiences of other entities. Your biological immune system is >entirely self-contained, making it robust and reliable in isolation. Actually, our immune systems no longer function in isolation, since the invention of vaccines. Our immune systems are now part of a system that includes all the research laboratories working on communicable diseases, as well as the health care delivery system. Sometimes this system provides an avenue for biologic attack, as for example the spread of AIDS through blood transfusions and (in the third world) unsterile needles. However it is a physically partitioned system and that provides a great deal of protection. By the same token I might well accept an ROM library implant in my head; it's the network connection I would reject. I want to control when that library changes. >computer code is so much easier to analyze and work with (compared >to protoplasm) that I think a great advantage accrues to the >defender. Ah, but so far the pathogens our bodies have had to deal with have been created by blind evolution, not by intelligent and hostile entities. -- Gord Deinstadt gdeinstadt@geovision.UUCP