This paper focuses on gradient-based backpropagation algorithms that use either a common adaptive learning rate for all weights or a separate adaptive learning rate for each weight. The learning-rate adaptation is based on descent techniques and estimates of the local constants that are obtained without additional error function and gradient evaluations. This paper proposes three algorithms to improve the different versions of backpropagation training in terms of both convergence rate and convergence characteristics, such as stable learning and robustness to oscillations. The new modification consists of a simple change in the error signal function. Experiments are conducted to compare and evaluate the convergence behavior of these gradient-based training algorithms with three training problems: XOR, encoding problem and character recognition, which are popular training problems.
In this paper, we report on experiments in which we used neural
networks for statistical anomaly intrusion detection systems. The five types of neural networks that we studied were: Perceptron; Backpropagation; Perceptron-Backpropagation-Hybrid; Fuzzy ARTMAP; and Radial-Based Function. We collected four separate data sets from different simulation scenarios, and these data sets were used to test various neural networks with different hidden neurons. Our results showed that the classification capabilities of BP and PBH outperform those of other neural networks.
Control chart pattern (CCP) recognition is important for monitoring
process environments to achieve appropriate control precisely and quickly and to produce high quality products. CCPs are represented by a large number of inputs. The principal component analysis (PCA) is an effective procedure for reducing a large input vector to a small vector.
This paper describes an efficient approach to reducing the inputs of the networks for CCP recognition with the use of PCA. The reason for applying PCA to CCP recognition is to provide simplicity for the networks and to speed up the training procedure of them. Multilayered perceptrons (MLP) are used and trained with the resilient-propagation (RP) and the backpropagation (BP) learning algorithms. The results show that PCA provides less cornplex neural network structure for accurate and faster training. This helps to achieve the CCP recognition precisely and accurately and rnight even help us to implement the recognition easily within the VLSI technologies for this application.
Control chart patterri (CCP) recognition is important for monitoring
process environments to achieve appropriate control precisely and quickly and to produce high quality products. CCPs are represented by a large number of inputs. The principal component analysis (PCA) is an effective procedure for redncing a large input vector to a small vector.
This paper describes an efficient approach to redncing the inputs of the networks for CCP recognition with the use of PCA. The reason for applying PCA to CCP recognition is to provide simplicity for the networks and to speed up the training procedure of them. Multilayered perceptrons (MLP) are used and trained with the resilient-propagation (RP) and the backpropagation (BP) learning algorithms. The results show that PCA provides less complex neural network structure for accurate and faster training. This helps to achieve the CCP recognition precisely and accurately and rnight even help us to implement the recognition easily within the VLSI technologies for this application.
It is generally accepted that most benchmark problems known today
can be solved by artificial neural networks with one single hidden layer. Networks with more than one hidden layer normally slow down learning dramatically. Furthermore, generalisation to new input patterns is generally better in small networks [1], [2], However, most benchmark problems only involve a small training data set which is normally discrete (such as binary values 0 and 1) in nature. The ability of single hidden layer supervised networks to solve problems with large and continuous type of data (e.g. most engineering problems) is virtually unknown. A fast learning method for solving continuous type problems has been proposed by Evans et al. [3]. However, the method is based on the Kohonen competitive, and ART unsupervised network models. In addition, almost every benchmark problem has the training set containing all possible input patterns, so there is no study of the generalisation behaviour of the network [4]. This study attempts to show that Single hidden layer supervised networks can be used to solve large and continuous type problems within measurable algorithmic complexities.
Backpropagation which uses gradient descent on the steepness of the sigmoid function (BPSA) has been widely studied (e.g. Kruschke et al. [1]). However, most of these studies only analysed the BPSA empirically where no adequate measurements of the network’s quality characteristics (e.g. efficiency and complexity) were given. This paper attempts to show that the BPSA is more efficient than the standard BPA by quantitatively comparing the convergence performance of both algorithms on several benchmark application problems. The convergence performance is measured by the values of the neural metrics [2] evaluated in the training process.