The previous chapter concentrated on multilayer
architectures with sigmoidal type units, both static (feedforward)
and dynamic. The present chapter introduces several additional
adaptive multilayer networks and their associated training procedures,
as well as some variations. The majority of the networks considered
here employ processing units which are not necessarily sigmoidal.
A common feature in these networks is their fast training as
compared to the backprop networks of the previous chapter. The
mechanisms leading to such increased training speed are emphasized.
All networks discussed in this chapter differ in
one or more significant ways from those in the previous chapter.
One group of networks employs units with localized receptive
fields, where units receiving direct input from input signals
(patterns) can only "see" a part of the input pattern.
Examples of such networks are the radial basis function network
and the cerebellar model articulation controller.
A second group of networks employs resource allocation.
These networks are capable of allocating units as needed during
training. This feature enables the network size to be determined
dynamically and eliminates the need for guessing the proper network
size. This resource allocating scheme is also shown to be the
primary reason for efficient training. Examples of networks in
this group are hyperspherical classifiers and the cascade-correlation
network.
The above two groups of networks mainly employ supervised
learning. Some of these networks may be used as function interpolators/approximators,
while others are best suited for classification tasks. The third
and last group of adaptive multilayer networks treated in this
chapter has the capability of unsupervised learning or clustering.
Here, two specific clustering nets are discussed: The ART1 network
and the autoassociative clustering network.
Throughout this Chapter, fundamental similarities
and differences among the various networks are stressed. In addition,
significant extensions of the above networks are pointed out and
the effects of these extensions on performance are discussed.
