Neural ordinary differential equations (NODE) present a new way of considering a deep residual network as a continuous structure by layer depth. However, it fails to overcome its representational limits, where it cannot learn all possible homeomorphisms of input data space, and therefore quickly saturates in terms of performance even as the number of layers increases. Here, we show that simply stacking Neural ODE blocks could easily improve performance by alleviating this issue. Furthermore, we suggest a more effective way of training neural ODE by using a time-evolving mixture weight on multiple ODE functions that also evolves with a separate neural ODE. We provide empirical results that are suggestive of improved performance over stacked as well as vanilla neural ODEs where we also confirm our approach can be orthogonally combined with recent advances in neural ODEs.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1109/TPAMI.2022.3145013 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Novel numerical and artificial neural computing with experimental validation towards unsteady micropolar nanofluid flow across a Riga plate.
Sci Rep
January 2025
Department of Mechanical Engineering, College of Engineering and Computer Sciences, Jazan University, P.O Box 45124, Jazan, Saudi Arabia.
Fluid flow across a Riga Plate is a specialized phenomenon studied in boundary layer flow and magnetohydrodynamic (MHD) applications. The Riga Plate is a magnetized surface used to manipulate boundary layer characteristics and control fluid flow properties. Understanding the behavior of fluid flow over a Riga Plate is critical in many applications, including aerodynamics, industrial, and heat transfer operations.
View Article and Find Full Text PDFExploring spiking neural networks for deep reinforcement learning in robotic tasks.
Sci Rep
December 2024
Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", Università di Bologna, 40126, Bologna, Italy.
Spiking Neural Networks (SNNs) stand as the third generation of Artificial Neural Networks (ANNs), mirroring the functionality of the mammalian brain more closely than their predecessors. Their computational units, spiking neurons, characterized by Ordinary Differential Equations (ODEs), allow for dynamic system representation, with spikes serving as the medium for asynchronous communication among neurons. Due to their inherent ability to capture input dynamics, SNNs hold great promise for deep networks in Reinforcement Learning (RL) tasks.
View Article and Find Full Text PDFInvestigation on the regular and chaotic dynamics of a ring network of five inertial Hopfield neural network: theoretical, analog and microcontroller simulation.
Cogn Neurodyn
December 2024
Department of Electronics and Communication Engineering, Vemu Institute of Technology, Chittoor, India.
The studies conducted in this contribution are based on the analysis of the dynamics of a homogeneous network of five inertial neurons of the Hopfield type to which a unidirectional ring coupling topology is applied. The coupling is achieved by perturbing the next neuron's amplitude with a signal proportional to the previous one. The system consists of ten coupled ODEs, and the investigations carried out have allowed us to highlight several unusual and rarely related dynamics, hence the importance of emphasizing them.
View Article and Find Full Text PDFTraining stiff neural ordinary differential equations with implicit single-step methods.
Chaos
December 2024
Department of Mechanical Engineering, University of California, Santa Barbara, Santa Barbara, California 93106, United States.
Stiff systems of ordinary differential equations (ODEs) are pervasive in many science and engineering fields, yet standard neural ODE approaches struggle to learn them. This limitation is the main barrier to the widespread adoption of neural ODEs. In this paper, we propose an approach based on single-step implicit schemes to enable neural ODEs to handle stiffness and demonstrate that our implicit neural ODE method can learn stiff dynamics.
View Article and Find Full Text PDFHeMiTo-dynamics: a characterisation of mammalian prion toxicity using non-dimensionalisation, linear stability and perturbation analyses.
Math Med Biol
November 2024
Mathematical Institute, University of Oxford, Andrew Wiles Building, Woodstock Road, OX2 6GG, Oxfordshire, United Kingdom.
Prion-like proteins play crucial parts in biological processes in organisms ranging from yeast to humans. For instance, many neurodegenerative diseases are believed to be caused by the production of prion-like proteins in neural tissue. As such, understanding the dynamics of prion-like protein production is a vital step toward treating neurodegenerative disease.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!