Such information is maintained with relative ease if it is relevant at each intermediate step it tends to be lost when intervening elements do not depend on it. We explore the conditions under which the network can carry information about distant sequential contingencies across intervening elements. When the network has a minimal number of hidden units, patterns on the hidden units come to correspond to the nodes of the grammar, although this correspondence is not necessary for the network to act as a perfect finite-state recognizer. When the network is trained with strings from a particular finite-state grammar, it can learn to be a perfect finite-state recognizer for the grammar. The network uses the pattern of activation over a set of hidden units from time-step t−1, together with element t, to predict element t + 1. We explore a network architecture introduced by Elman (1988) for predicting successive elements of a sequence.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |