Upper bounds on the node numbers of hidden layers in MLPs

Title:

Upper bounds on the node numbers of hidden layers in MLPs

Creator:

Liu, Jiang, Ni, Feng, Du, Mingjun, Zhang, Xuyang, Que, Zhongli, and Song, Shihang

Identifier:

https://cdk.lib.cas.cz/client/handle/uuid:024cfe92-806b-4011-a140-fb9265238f91
uuid:024cfe92-806b-4011-a140-fb9265238f91
doi:10.14311/NNW.2021.31.016

Subject:

feedforward neural network, hidden node, multilayer perceptron (MLP), and upper bound

Type:

model:article and TEXT

Format:

bez média and svazek

Description:

It is one of the fundamental and challenging problems to determine the node numbers of hidden layers in neural networks. Various efforts have been made to study the relations between the approximation ability and the number of hidden nodes of some specific neural networks, such as single-hidden-layer and two-hiddenlayer feedforward neural networks with specific or conditional activation functions. However, for arbitrary feedforward neural networks, there are few theoretical results on such issues. This paper gives an upper bound on the node number of each hidden layer for the most general feedforward neural networks called multilayer perceptrons (MLP), from an algebraic point of view. First, we put forward the method of expansion linear spaces to investigate the algebraic structure and properties of the outputs of MLPs. Then it is proved that given k distinct training samples, for any MLP with k nodes in each hidden layer, if a certain optimization problem has solutions, the approximation error keeps invariant with adding nodes to hidden layers. Furthermore, it is shown that for any MLP whose activation function for the output layer is bounded on R, at most k hidden nodes in each hidden layer are needed to learn k training samples.

Language:

English

Rights:

http://creativecommons.org/licenses/by-nc-sa/4.0/
policy:public

Coverage:

297-309

Source:

Neural network world: international journal on neural and mass-parallel computing and information systems | 2021 Volume:31 | Number:4

Harvested from:

CDK

Metadata only:

false