Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!bloom-beacon!bu-cs!bucsb!adverb
From: adverb@bucsb.UUCP (Josh Krieger)
Newsgroups: comp.ai.neural-nets
Subject: Re: request for philosophic reactions to connectionism
Summary: Comments on ART
Keywords: ART, Grossberg
Message-ID: <1@bucsb.UUCP>
Date: 23 Apr 89 03:56:50 GMT
References: <370@eurtrx.UUCP| <18496@gatech.edu> <7894@phoenix.Princeton.EDU> <18504@gatech.edu> <7903@phoenix.Princeton.EDU>
Reply-To: adverb@bucsb.bu.edu (Josh Krieger)
Followup-To: comp.ai.neural-nets
Organization: Boston Univ Comp. Sci.
Lines: 90

Please stop dismissing ART out of ignorance!

Grossberg's papers are exceptionally difficult to understand because
each discovery is dependent on the "minimal" biological building
blocks discovered in the past 30 years. It would be impossible
to understand microprocessor intricacies with only basic knowledge
of transistors; so don't expect to understand Grossberg's work
without a little research and time. ART is a beautiful discovery.

A good introduction might be trying to read "How does a Brain Build
a Cognitive Code", it's reprinted in Anderson and Rosenthall's
"Neurocomputing" and Grossberg's "Studies of Mind and Brain". It will
take about 4 readings to get the information solid. But don't
be frightened by it.

What follows is meant to be a very simple explanation of how
ART I works. The description is not complete; however, it gives
an idea of the inherent power in the model.

ART consists of 2 subsytems: 1) An attentional subsystem
and 2) An orienting subsytem:


            Attentional Subsystem       Orienting Subsystem

              **** Layer 2 ****
               |            <-   <-----\
	       |             |           \
               ->            |             \
              **** Layer 1 ****   --->       ****
               ! ! ! ! ! ! ! !
                   Input

The attentional subsystem will accept an input pattern
into Layer 1. Bottom-up signals will filter the pattern
into Layer 2 where noise is removed and features are enhanced.
All the cells in Layer 2 will compete and the one which
wins the competition produces a "learned expectancy" or top-down
signal back to layer 1. If the expectancy is not an accurate match
of the input pattern (there is a definition of accurate), then
the orienting subsystem is notified by the attentional subsystem.
The orienting subsystem proceeds to send an arousal burst back
to the attentional subsystem which resets the "category cell"
that produced the expectancy. The process is repeated until
a cell is found which produces an accurate top-down
expectancy of the input. In other words, one cell in layer 2
will represent the exemplar of particular category
and that cell will not be recoded to a different category
even if a continuous set of input patterns are presented to the
network. If a perfect match is not found, a category cell in
layer 2 is chosen and Bottom-up and Top-down activations reverberate
(or resonate) until both the bottom-up connections and the
top-down connections have learned the input pattern.

The advantages:

 1) Once a pattern is learned retrieval time occurs in one
    feedforward pass.
 2) ART has a vigilance level which allows it to
    place input patterns into categories based upon course
    or fine distinctions (Course destinctions would be
    classifying the letters C and D in the same category, while fine
    would involve placing them in different categories.
 3) Old learning is not washed away by the
    "blooming, buzzing confusion", of the real world.
    ART is stable to old learning yet plastic to new information.
 4) ART will converge regardless of the parameter settings.
    There is no tweaking of learning rates.
 5) ART has a vast amount of solid psychological and biological
    evidence for its existence.

Grossberg himself writes:

"The success of these circuits in organizing large interdisciplinary
data bases suggests that they will remain building blocks in any
future theory that supplants the present stage of understanding."

In other words, ART will not be replaced. It will be added to.

"Studies of Mind and Brain" is an incredibly difficult collection to get
through. "The Adapative Brain" is much easier, although it still
relies on previous concepts. Grossberg's work is THE most important
research available in neural networks and to ignore it would be
the equivalent, putting things into a historical perspective,
of having disregarded the potential significance of the transistor
in the 1940's.

One last comment: Backprop and ART are apples and oranges!

-- Josh Krieger (adverb%bucsx.bu.edu@bu-cs.bu.edu)