Simple recurrent network trained by RTRL and exetnded Kalman filter algorithms

Title:

Simple recurrent network trained by RTRL and exetnded Kalman filter algorithms

Creator:

Čerňanský, Michal and Beňušková, Ľubica

Identifier:

https://cdk.lib.cas.cz/client/handle/uuid:68781331-8344-443c-94a3-107056baf0bd
uuid:68781331-8344-443c-94a3-107056baf0bd

Subject:

recurrent neural networks, extended Kalman filter, next symbol prediction, and recursive languages

Type:

model:article and TEXT

Format:

bez média and svazek

Description:

Recurrent neural networks (RNNs) have much larger potential than classical feed-forward neural networks. Their output responses depend also on the time position of a given input and they can be successfully used in spatio-temporal task Processing. RNNs are often used in the cognitive science community to process symbol sequences that represent various natural language structures. Usually they are trained by common gradient-based algorithms such as real time recurrent learning (RTRL) or backpropagation through time (BPTT). This work compares the RTRL algorithm that represents gradient based approaches with extended Kalman filter (EKF) methodology adopted for training the Elman’s simple recurrent network (SRN). We used data sets containing recursive structures inspired by studies of cognitive science community and trained SRN for the next symbol prediction task. The EKF approach, although computationally more expensive, shows higher robustness and the resulting next symbol prediction performance is higher.

Language:

English

Rights:

http://creativecommons.org/publicdomain/mark/1.0/
policy:public

Source:

Neural network world: international journal on neural and mass-parallel computing and information systems | 2003 Volume:13 | Number:3

Harvested from:

CDK

Metadata only:

false