Xref: utzoo comp.ai:3312 sci.lang:4067 Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!rutgers!elbereth.rutgers.edu!harnad From: harnad@elbereth.rutgers.edu (Stevan Harnad) Newsgroups: comp.ai,sci.lang Subject: Re: Categorization Summary: Feature detection, symbolic rules and connectionism Message-ID: Date: 5 Feb 89 05:31:13 GMT References: <15585@cisunx.UUCP> <3200@uhccux.uhcc.hawaii.edu> Organization: Rutgers Univ., New Brunswick, N.J. Lines: 116 anwst@cisunx.UUCP (Anders N. Weinstein) of Univ. of Pittsburgh, Comp & Info Sys wrote: " [We must distinguish] the normative question of which things are " *correctly* classified as birds or even numbers, and the descriptive " question of how in fact our neural machinery functions to enable us to " so classify things. I agree also with Harnad that psychology ought to " keep its focus on the latter and not the former of these questions. A kind of "correctness" factor does figure in the second question too: To model how people categorize things we have to have data on what inputs they categorize as members of what categories, according to what constraints on MIScategorization. However, it's certainly not an ontological correctness that's at issue, i.e., we're not concerned with what the things people categorize really ARE "sub specie aeternitatis": We're just concerned with what people get provisionally right and wrong, under the constraints of the sample they've encountered so far and the feedback they've so far received from the consequences of miscategorization. " I think Harnad errs... that reliable categorization *must* be " interestingly describable as application of some (perhaps complex) rule " in "featurese" (for some appropriate set of detectable features)... " Limiting ourselves (as I think we must) to quick and automatic " observational classification... If... the effects of context on such tasks " are minimal... there must be within us some isolable module which can " take sensory input and produce a one bit yes-or-no output for category " membership... But how does it follow that such a device must be " describable as applying some *rule*? Any physical object in the world " could be treated as a recognition device for something by interpreting " some of its states as "inputs" and some as "yes-or-no responses." But " intuitively, it looks like not every such machine is usefully described " as applying a rule in this way. In particular, this certainly doesn't " seem a natural way of describing connectionist pattern recognizers. So " why couldn't it turn out that there is just no simpler description of " the "rule" for certain category membership than: whatever a machine of " a certain type recognizes? I don't care whether or not the internal basis for a machine's feature-detecting and categorizing success is described by us as a "rule" (though I suspect it can always be described that way). I don't even care whether or not the internal basis consists of an explicit representation of a symbolic rule that is actually "applied" (in fact, according to my theory, such symbolic representations of categories would first have to be grounded in prior nonsymbolic representations). A connectionist feature-detector would be perfectly fine with me; I even suggest in my book that that would be a natural (and circumscribed) role for a connectionist module to play in a category representation system (if it can actually deliver the goods). To rehabilitate the "classical" view I've been trying to rescue from well over a decade of red herrings and incoherent criticism all I need to re-establish is that where there is reliable, correct, all-or-none categorization performance, there must surely exist detectable features in the input that are actually detected by the categorizing device as a basis for its successful categorization performance. I think this should be self-evident to anyone who is mindful of the obvious facts about our categorization performance capacity and is not in the grip of a California theory (and does not believe in magic). The so-called "classical" view is only that features must EXIST in the inputs that we are manifestly able to sort and label, and that these features are actually DETECTED and USED to generate our successful performance. The classical view is not committed to internal representations of rules symbolically describing the features in "featurese" or operating on symbolic descriptions of features. That's another issue. (According to my own theory, symbolic "featurese" itself, like all abstract category labels in the "language of thought," must first be grounded in nonsymbolic, sensory categories and their nonsymbolic, sensory features.) [By the way, I don't think there's really a problem with sorting out which devices are actually categorizing and which ones aren't. Do you, really? That sounds like a philosopher's problem only. (If what you're worried about is whether the categorizer really has a mind, then apply my Total Turing Test -- require it to have all of our robotic and linguistic capacities.) Nor does "whatever a machine of a certain type recognizes" sound like a satisfactory answer to the "question of how in fact our neural machinery functions to enable us to so classify things." You have to say what features it detects, and HOW.] [Related to the last point, Greg Lee (lee@uhccux.uhcc.hawaii.edu), University of Hawaii, had added, concerning connectionist feature-detectors: "If you don't understand how the machine works, how can you give a rule?" I agree that the actual workings of connectionist black boxes need more analysis, but to a first approximation the answer to the question of how they work (if and when they work) is: "they learn features by sampling inputs, with feedback about miscategorization, `using' back-prop and the delta rule." And that's certainly a lot better than nothing. A fuller analysis would require specifying what features they're detecting, and how they arrived at them on the available data, as constrained by back-prop and the delta rule. There's no need whatsoever for any rules to be explicitly "represented" in order to account fully for their success, however. -- In any case, connectionist black boxes apparently do not settle the classical/nonclassical matter one way or the other, as evidenced by the fact that there seems to be ample room for them in both nonclassical approaches (e.g., Lakoff's) and classical ones (e.g., mine).] I also see no reason to limit our discussion to "quick, automatic, observational" categorization; it applies just as much to slow perceptual pattern learning and, with proper grounding, to abstract, nonperceptual categorization too (although here is where explicitly represented symbolic rules [in "featurese"?] do play more of a role, according to my grounding theory). And I think context effects are rarely "minimal": All categorization is provisional and approximate, dependent on the context of confusable alternatives so far sampled, and the consequences (so far) of miscategorizing them. -- Stevan Harnad INTERNET: harnad@confidence.princeton.edu harnad@princeton.edu srh@flash.bellcore.com harnad@elbereth.rutgers.edu harnad@princeton.uucp BITNET: harnad@pucc.bitnet CSNET: harnad%princeton.edu@relay.cs.net (609)-921-7771