Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!seismo!rutgers!princeton!mind!harnad From: harnad@mind.UUCP (Stevan Harnad) Newsgroups: comp.ai,comp.cog-eng Subject: Re: The symbol grounding problem (Part 1) Message-ID: <768@mind.UUCP> Date: Fri, 22-May-87 11:22:49 EDT Article-I.D.: mind.768 Posted: Fri May 22 11:22:49 1987 Date-Received: Sat, 23-May-87 15:46:09 EDT References: <764@mind.UUCP> Organization: Cognitive Science, Princeton University Lines: 102 Keywords: Color categories, A/D filters, learning Summary: The difference between iconic and categorical representations Xref: mnetor comp.ai:440 comp.cog-eng:101 This is part 1 of a response to a longish exchange from Rik Belew who asks: > ... [1] what evidence causes you to postulate iconic and categorical > representations as being distinct?... Apart from a relatively few > cognitive phenomena (short-term sensory storage, perhaps mental > imagery), I am aware of little evidence of "continuous, isomorphic > analogues of the sensory surfaces" [your "iconic" representations]. > [2] I see great difficulty in distinguishing between such > representations and "constructive A/D filters [`categorical' > representations] which preserve the invariant sensory features" based > simply on performance at any particular task. More generally, could > you [3] motivate your ``subserve'' basis for classifying cognitive > representations. [1] First of all, short-term sensory storage does not seem to constitute *little* evidence but considerable evidence. The tasks we can perform after a stimulus is no longer present (such as comparing and matching) force us to infer that there exist iconic traces. The alternative hypthesis that the information is already a symbolic description at this stage is simply not parsimonious and does not account for all the data (e.g., Shepard's mental rotation effects). These short-term effects do suggest that iconic representations may only be temporary or transient, and that is entirely compatible with my model. Something permanent is also going on, however, as the sensory exposure studies suggest: Even if iconic traces are always stimulus-bound and transient, they seem to have a long-term substrate too, because their acuity and reliability increases with experience. I would agree that the subjective phenomenology of mental imagery is very weak evidence for long-term icons, but successful performance on some perceptual tasks drawing on long-term memory is at least as economically explained by the hypothesis that the icons are still accessible as by the alternative that only symbolic descriptions are being used. In my model, however, most long-term effects are mediated by the categorical representations rather than the iconic ones. Iconic representations are hypothesized largely to account for short-term perceptual performance (same/difference judgment, relative comparisons, similarity judgments, mental rotation, etc.). They are also, of course, more compatible with subjective phenomenology (memory images seem to be more like holistic sensory images than like selective feature filters or symbol strings). [2] The difference between isomorphic iconic representations (IRs) and selective invariance filters (categorical representations, CRs) is quite specific, although I must reiterate that CRs are really a special form of "micro-icon." They are still sensory, but they are selective, discarding most of the sensory variation and preserving only the features that are invariant *within a specific context of confusable alternatives*. (The key to my approach is that identifying or categorizing something is never an *absolute* task but a relative, context-dependent one: "What's that?" "Compared to What?") The only "features" preserved in a CR are the ones that will serve as a reliable basis for sorting the instances one has sampled into their respective categories (as learned from feedback indicating correct or incorrect categorizing). The "context" (of confusable alternatives), however, is not a short-term phenomenon. Invariant features are provisional, and always potentially revisable, but they are parts of a stable, long-term category-representational system, one that is always being extended and updated on the basis of new categorization tasks and samples. It constitutes an ever-tightening approximation. So the difference between IRs and CRs ("constructive A/D filters") is that IRs are context-independent, depending only on the comparison of raw sensory configurations and on any transformations that rely on isomorphism with the unfiltered sensory configuration, whereas IRs are context-dependent and depend on what confusable alternatives have been sampled and must then be reliably identified in isolation. The features on which this successful categorization is based cannot be the holistic configural ones, which blend continuously into one another; they are features specifically selected and abstracted to subserve reliable categorization (within the context of alternatives sampled to date). They may even be "constructive" features, in the sense that they are picked out by performing an active operation -- sensory, comparative or even logical -- on the sensory input. Apart from this invariant basis for categorization (let's call these selectively abstracted features "micro-iconic") all the rest of the iconic information is discarded from the category filter. [3] Having said all this, it is easy to motivate my "subserve" as you request: IRs are the representations that subserve ( = are required in order to generate successful performance on) tasks that call for holistic sensory comparisons and isomorphic transformations of the unfiltered sensory trace (e.g., discrimination, matching, similarity judgment) and CRs are the representations required to generate successsful performance on tasks that call for reliable identification of confusable alternatives presented in isolation. As a bonus, the latter provide the grounding for a third representational system, symbolic representations (SRs), whose elementary symbols are the labels of the bounded categories picked out by the CRs and "fleshed out" by the IRs. These elementary symbols can then be rulefully combined and recombined into symbolic descriptions which, in virtue of their reducibility to grounded nonsymbolic representations, can now refer to, describe, predict and explain objects and events in the world. -- Stevan Harnad (609) - 921 7771 {bellcore, psuvax1, seismo, rutgers, packard} !princeton!mind!harnad harnad%mind@princeton.csnet harnad@mind.Princeton.EDU