In recent years, gradient-based LSTM recurrent neural networks (RNNs) solved many previously RNN-unlearnable tasks. Sometimes, however, gradient information is of little use for training RNNs, due to numerous local minima. For such cases, we present a novel method: EVOlution of systems with LINear Outputs (Evolino). Evolino evolves weights to the nonlinear, hidden nodes of RNNs while computing optimal linear mappings from hidden state to output, using methods such as pseudo-inverse-based linear regression. If we instead use quadratic programming to maximize the margin, we obtain the first evolutionary recurrent support vector machines. We show that Evolino-based LSTM can solve tasks that Echo State nets (Jaeger, 2004a) cannot and achieves higher accuracy in certain continuous function generation tasks than conventional gradient descent RNNs, including gradient-based LSTM.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1162/neco.2007.19.3.757 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Optimized memory augmented graph neural network-based DoS attacks detection in wireless sensor network.
Network
October 2024
Department of Computer Science and Engineering, Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Chennai, India.
Wireless Sensor Networks (WSNs) are mainly used for data monitoring and collection purposes. Usually, they are made up of numerous sensor nodes that are utilized to gather data remotely. Each sensor node is small and inexpensive.
View Article and Find Full Text PDFBack to Normal? Harnessing Long Short-term Memory Network to Examine the Associations Between Adolescent Social Interactions and Depressive Symptoms During Different Stages of COVID-19.
Res Child Adolesc Psychopathol
October 2024
Technion - Israel Institute of Technology, Faculty of Medicine, Haifa, Israel.
Adolescence is a developmental period in which social interactions are critical for mental health. While the onset of COVID-19 significantly disrupted adolescents' social environments and mental health, it remains unclear how adolescents have adapted to later stages of the pandemic. We harnessed a machine learning architecture of Long Short-Term Memory recurrent networks (LSTM) with gradient-based feature importance, to model the association among daily social interactions and depressive symptoms during three stages of the pandemic.
View Article and Find Full Text PDFInterpretable LSTM model reveals transiently-realized patterns of dynamic brain connectivity that predict patient deterioration or recovery from very mild cognitive impairment.
Comput Biol Med
July 2023
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA.
Alzheimer's Disease (AZD) is a neurodegenerative disease for which there is now no known effective treatment. Mild cognitive impairment (MCI) is considered a precursor to AZD and affects cognitive abilities. Patients with MCI have the potential to recover cognitive health, can remain mildly cognitively impaired indefinitely or eventually progress to AZD.
View Article and Find Full Text PDFVision-Based HAR in UAV Videos Using Histograms and Deep Learning Techniques.
Sensors (Basel)
February 2023
School of Computer Science and Engineering, VIT-AP University, Amaravati 522237, India.
Activity recognition in unmanned aerial vehicle (UAV) surveillance is addressed in various computer vision applications such as image retrieval, pose estimation, object detection, object detection in videos, object detection in still images, object detection in video frames, face recognition, and video action recognition. In the UAV-based surveillance technology, video segments captured from aerial vehicles make it challenging to recognize and distinguish human behavior. In this research, to recognize a single and multi-human activity using aerial data, a hybrid model of histogram of oriented gradient (HOG), mask-regional convolutional neural network (Mask-RCNN), and bidirectional long short-term memory (Bi-LSTM) is employed.
View Article and Find Full Text PDFAir Quality Prediction Using the Fractional Gradient-Based Recurrent Neural Network.
Comput Intell Neurosci
December 2022
Birla Institute of Technology and Science Pilani, Pilani, Rajasthan, India.
In this study, the air quality index (AQI) of Indian cities of different tiers is predicted by using the vanilla recurrent neural network (RNN). AQI is used to measure the air quality of any region which is calculated on the basis of the concentration of ground-level ozone, particle pollution, carbon monoxide, and sulphur dioxide in air. Thus, the present air quality of an area is dependent on current weather conditions, vehicle traffic in that area, or anything that increases air pollution.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!