AI Article Synopsis
- The study demonstrates that neural networks with widths equal to or less than the input dimension have unbounded connected components in their decision regions.
- This finding applies to both continuous and strictly monotonic activation functions, including the ReLU function.
- The results provide additional insights into the approximation capabilities and connectivity of decision regions in narrow neural networks, validated through numerical experiments.
Article Abstract
We show that for neural network functions that have width less or equal to the input dimension all connected components of decision regions are unbounded. The result holds for continuous and strictly monotonic activation functions as well as for the ReLU activation function. This complements recent results on approximation capabilities by Hanin and Sellke (2017) and connectivity of decision regions by Nguyen et al. (2018) for such narrow neural networks. Our results are illustrated by means of numerical experiments.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1016/j.neunet.2021.02.024 | DOI Listing |
Publication Analysis
Top Keywords
decision regions
12
neural networks
8
regions narrow
4
narrow deep
4
deep neural
4
networks neural
4
neural network
4
network functions
4
functions width
4
width equal
4
Similar Publications
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!