6. Adaptive Multilayer Neural Networks II

6.0 Introduction

The previous chapter concentrated on multilayer architectures with sigmoidal type units, both static (feedforward) and dynamic. The present chapter introduces several additional adaptive multilayer networks and their associated training procedures, as well as some variations. The majority of the networks considered here employ processing units which are not necessarily sigmoidal. A common feature in these networks is their fast training as compared to the backprop networks of the previous chapter. The mechanisms leading to such increased training speed are emphasized.

All networks discussed in this chapter differ in one or more significant ways from those in the previous chapter. One group of networks employs units with localized receptive fields, where units receiving direct input from input signals (patterns) can only "see" a part of the input pattern. Examples of such networks are the radial basis function network and the cerebellar model articulation controller.

A second group of networks employs resource allocation. These networks are capable of allocating units as needed during training. This feature enables the network size to be determined dynamically and eliminates the need for guessing the proper network size. This resource allocating scheme is also shown to be the primary reason for efficient training. Examples of networks in this group are hyperspherical classifiers and the cascade-correlation network.

The above two groups of networks mainly employ supervised learning. Some of these networks may be used as function interpolators/approximators, while others are best suited for classification tasks. The third and last group of adaptive multilayer networks treated in this chapter has the capability of unsupervised learning or clustering. Here, two specific clustering nets are discussed: The ART1 network and the autoassociative clustering network.

Throughout this Chapter, fundamental similarities and differences among the various networks are stressed. In addition, significant extensions of the above networks are pointed out and the effects of these extensions on performance are discussed.

Goto [6.1] [6.2] [6.3] [6.4] [6.5]

Back to the Table of Contents

Back to Main Menu