This chapter started by introducing the radial basis
function (RBF) network as a two layer feedforward net employing
hidden units with locally-tuned response characteristics. This
network model is motivated by biological nervous systems as well
as by early results on the statistical and approximation properties
of radial basis functions. The most natural application of RBF
networks is in approximating smooth, continuous multivariate functions
of few variables.
RBF networks employ a computationally efficient
training method that decouples learning at the hidden layer from
that at the output layer. This method uses a simple clustering
(competitive learning) algorithm to locate hidden unit receptive
field centers. It also uses the LMS or delta rule to adjust the
weights of the output units.
These networks have comparable prediction/approximation
capabilities to those of backprop networks, but train by orders
of magnitude faster. Another advantage of RBF nets is their lower
"false-positive" classification error rates. However,
by a factor of at least one order of magnitude, RBF nets require
more training data and more hidden units compared to backprop
nets for achieving the same level of accuracy.
Two major variations to the RBF network were given
which lead to improved accuracy (though, at the cost of reduced
training speed). The first variation involved replacing the k-means-clustering-based
training method for locating hidden unit centers by a "soft
competition" clustering method. The second variation to
the RBF net substitutes semilocal activation units for the local
activation hidden units, and employs gradient descent-based learning
for adjusting all unit parameters.
The CMAC is another example of a localized receptive
field net which was considered in this chapter. The CMAC was
originally developed as a model of the cerebellum. This network
model shares several of the features of the RBF net such as fast
training and the need for a large number of localized receptive
field hidden units for accurate approximation. It also has common
features (and limitations) to those found in classical perceptrons
(e.g., Rosenblatt's perceptron). One distinguishing feature of
the CMAC, though, is its built-in capability of local generalization.
The CMAC has been successfully applied in the control of robot
Three unit-allocating adaptive multilayer feedforward
networks were also described in this chapter. The first two networks
belong to the class of hyperspherical classifiers. They employ
hidden units with adaptive localized receptive fields. They also
have sparse interconnections between the hidden units and the
output units. These networks are easily capable of forming arbitrarily
complex decision boundaries, with rapid training times. In fact,
for one of these networks (PTC net), the training time was shown
to be of polynomial complexity.
The third unit allocating network (cascade-correlation
net) differs from all previous networks in its ability to build
a deep net of cascaded units and in its ability to utilize more
than one type of hidden units, co-existing in the same network.
The motivation behind unit-allocating nets is two fold: (1) The
elimination of the guesswork involved in determining the appropriate
number of hidden units (network size) for a given task, and (2)
Finally, two examples of dynamic multilayer clustering
networks are discussed: The ART1 net and the autoassociative clustering
net. These networks are intended for tasks involving data clustering
and prototype generation. The ART1 net is characterized by its
on-line capability of clustering binary patterns, its stability,
and its ability to follow nonstationary input distributions.
Generalizations of this network allow the extension of these desirable
characteristics to the clustering of analog patterns. These ART
networks are biologically motivated and were developed as possible
models of cognitive phenomena in humans and animals.
The second clustering net is motivated by "concept
forming" cognitive models. It is based on two interrelated
mechanisms: prototype formation and prototype extraction. A slightly
modified backprop training method is employed in a customized
autoassociative net of sigmoid units in an attempt to estimate
and encode cluster prototypes in such a way that they become attractors
of a dynamical system. This dynamical system is formed by taking
the trained feedforward net and feeding its output back to its
input. Results of simulations involving data clustering with
these nets are given. In particular, results of motor unit potential
(MUP) prototype extraction and noisy/distorted MUP categorization
(clustering) for a real EMG signal are presented.
It is hoped that the different network models presented
in this chapter, and the motivations for developing them give
the reader an appreciation of the diversity and richness of these
networks, and the way their development has been influenced by
biological, cognitive, and/or statistical models.
Back to the Table of Contents
Back to Main Menu