10 Recurrent Neural Networks Design And Applications, Neural Network, Artificial Neural Networks
[ Pobierz całość w formacie PDF ]
RECURRENT
NEURAL
NETWORKS
Edited by
L.R. Medsker
Departments of Physics and Computer
Science and Information Systems
American University
Washington, D.C.
L.C. Jain
Knowledge-Based Intelligent Engineering Systems Centre
Faculty of Information Technology
Director/Founder, KES
University of South Australia, Adelaide
The Mawson Lakes, SA
Australia
CRC Press
Boca Raton London New York Washington, D.C.
Design and Applications
PREFACE
Recurrent neural networks have been an interesting and important part of
neural network research during the 1990's. They have already been applied to a
wide variety of problems involving time sequences of events and ordered data
such as characters in words. Novel current uses range from motion detection and
music synthesis to financial forecasting.
This book is a summary of work on
recurrent neural networks and is exemplary of current research ideas and
challenges in this subfield of artificial neural network research and development.
By sharing these perspectives, we hope to illuminate opportunities and
encourage further work in this promising area.
Two broad areas of importance in recurrent neural network research, the
architectures and learning techniques, are addressed in every chapter.
Architectures range from fully interconnected to partially connected networks,
including recurrent multilayer feedforward. Learning is a critical issue and one
of the primary advantages of neural networks. The added complexity of learning
in recurrent networks has given rise to a variety of techniques and associated
research projects. A goal is to design better algorithms that are both
computationally efficient and simple to implement.
Another broad division of work in recurrent neural networks, on which this
book is structured, is the design perspective and application issues. The first
section concentrates on ideas for alternate designs and advances in theoretical
aspects of recurrent neural networks. Some authors discuss aspects of improving
recurrent neural network performance and connections with Bayesian analysis
and knowledge representation, including extended neuro-fuzzy systems. Others
address real-time solutions of optimization problems and a unified method for
designing optimization neural network models with global convergence.
The second section of this book looks at recent applications of recurrent
neural networks. Problems dealing with trajectories, control systems, robotics,
and language learning are included, along with an interesting use of recurrent
neural networks in chaotic systems. The latter work presents evidence for a
computational paradigm that has higher potential for pattern capacity and
boundary flexibility than a multilayer static feedforward network. Other
chapters examine natural language as a dynamic system appropriate for
grammar induction and language learning using recurrent neural networks.
Another chapter applies a recurrent neural network technique to problems in
controls and signal processing, and other work addresses trajectory problems
and robot behavior.
The next decade should produce significant improvements in theory and
design of recurrent neural networks, as well as many more applications for the
creative solution of important practical problems. The widespread application of
recurrent neural networks should foster more interest in research and
development and raise further theoretical and design questions.
ACKNOWLEDGMENTS
The editors thank Dr. R. K. Jain, University of South Australia, for his assistance
as a reviewer. We are indebted to Samir Unadkat and Mãlina Ciocoiu for their
excellent work formatting the chapters and to others who assisted: Srinivasan
Guruswami and Aravindkumar Ramalingam. Finally, we thank the chapter
authors who not only shared their expertise in recurrent neural networks, but also
patiently worked with us via the Internet to create this book. One of us (L.M.)
thanks Lee Giles, Ashraf Abelbar, and Marty Hagan for their assistance and
helpful conversations and Karen Medsker for her patience, support, and
technical advice.
THE EDITORS
Larry Medsker is a Professor of Physics and Computer Science at American
University. His research involves soft computing and hybrid intelligent systems
that combine neural network and AI techniques. Other areas of research are in
nuclear physics and data analysis systems. He is the author of two books:
Hybrid
Neural Network and Expert Systems
(1994) and
Hybrid Intelligent Systems
(1995). He co-authored with Jay Liebowitz another book on
Expert Systems and
Neural Networks
(1994). One of his current projects applies intelligent web-
based systems to problems of knowledge management and data mining at the
U.S. Department of Labor. His Ph.D. in Physics is from Indiana University, and
he has held positions at Bell Laboratories, University of Pennsylvania, and
Florida State University. He is a member of the International Neural Network
Society, American Physical Society, American Association for Artificial
Intelligence, IEEE, and the D.C. Federation of Musicians, Local 161-710.
L.C. Jain is a Director/Founder of the Knowledge-Based Intelligent Engineering
Systems (KES) Centre, located in the University of South Australia. He is a
fellow of the Institution of Engineers Australia. He has initiated a postgraduate
stream by research in the Knowledge-Based Intelligent Engineering Systems
area. He has presented a number of keynote addresses at International
Conferences on Knowledge-Based Systems, Neural Networks, Fuzzy Systems
and Hybrid Systems. He is the Founding Editor-in-Chief of the
International
Journal of Knowledge-Based Intelligent Engineering Systems
and served as an
Associate Editor of the
IEEE Transactions on Industrial Electronics
. Professor
Jain was the Technical chair of the ETD2000 International Conference in 1995,
Publications Chair of the Australian and New Zealand Conference on Intelligent
Information Systems in 1996 and the Conference Chair of the International
Conference on Knowledge-Based Intelligent Electronic Systems in 1997, 1998
and 1999. He served as the Vice President of the Electronics Association of
South Australia in 1997. He is the Editor-in-Chief of the International Book
Series on Computational Intelligence, CRC Press USA. His interests focus on
the applications of novel techniques such as knowledge-based systems, artificial
neural networks, fuzzy systems and genetic algorithms and the application of
these techniques.
Table of Contents
Chapter 1
Introduction
Samir B. Unadkat, Mãlina M. Ciocoiu and Larry R. Medsker
I.
Overview
A. Recurrent Neural Net Architectures
B. Learning in Recurrent Neural Nets
II.
Design Issues And Theory
A. Optimization
B. Discrete-Time Systems
C. Bayesian Belief Revision
D. Knowledge Representation
E. Long-Term Dependencies
III.
Applications
A. Chaotic Recurrent Networks
B. Language Learning
C. Sequential Autoassociation
D. Trajectory Problems
E. Filtering And Control
F. Adaptive Robot Behavior
IV.
Future Directions
Chapter 2
Recurrent Neural Networks for Optimization:
The State of the Art
Youshen Xia and Jun Wang
I.
Introduction
II.
Continuous-Time Neural Networks for QP and LCP
A. Problems and Design of Neural Networks
B. Primal-Dual Neural Networks for LP and QP
C. Neural Networks for LCP
III.
Discrete-Time Neural Networks for QP and LCP
A. Neural Networks for QP and LCP
B. Primal-Dual Neural Network for Linear Assignment
IV.
Simulation Results
V.
Concluding Remarks
[ Pobierz całość w formacie PDF ]