Connections Between Numerical Algorithms for PDEs and Neural Networks.

We investigate numerous structural connections between numerical algorithms for partial differential equations (PDEs) and neural architectures. Our goal is to transfer the rich set of mathematical foundations from the world of PDEs to neural networks. Besides structural insights, we provide concrete examples and experimental evaluations of the resulting architectures.

Designing rotationally invariant neural networks from PDEs and variational methods.

Partial differential equation models and their associated variational energy formulations are often rotationally invariant by design. This ensures that a rotation of the input results in a corresponding rotation of the output, which is desirable in applications such as image analysis. Convolutional neural networks (CNNs) do not share this property, and existing remedies are often complex.

Ray tracing in a finite-element domain using nodal basis functions.

A method is presented for tracing rays through a medium discretized as finite-element volumes. The ray-trajectory equations are cast into the local element coordinate frame, and the full finite-element interpolation is used to determine instantaneous index gradient for the ray-path integral equation. The finite-element methodology is also used to interpolate local surface deformations and the surface normal vector for computing the refraction angle when launching rays into the volume, and again when rays exit the medium.