Nyströmformer: A Nystöm-based Algorithm for Approximating Self-Attention.

Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences - a topic being actively studied in the community.

Performing Group Difference Testing on Graph Structured Data From GANs: Analysis and Applications in Neuroimaging.

Generative adversarial networks (GANs) have emerged as a powerful generative model in computer vision. Given their impressive abilities in generating highly realistic images, they are also being used in novel ways in applications in the life sciences. This raises an interesting question when GANs are used in scientific or biomedical studies.

Scaling Recurrent Models via Orthogonal Approximations in Tensor Trains.

Modern deep networks have proven to be very effective for analyzing real world images. However, their application in medical imaging is still in its early stages, primarily due to the large size of three-dimensional images, requiring enormous convolutional or fully connected layers - if we treat an image (and not image patches) as a sample. These issues only compound when the focus moves towards longitudinal analysis of 3D image volumes through recurrent structures, and when a point estimate of model parameters is insufficient in scientific applications where a reliability measure is necessary.

On Training Deep 3D CNN Models with Dependent Samples in Neuroimaging.

There is much interest in developing algorithms based on 3D convolutional neural networks (CNNs) for performing regression and classification with brain imaging data and more generally, with biomedical imaging data. A standard assumption in learning is that the training samples are independently drawn from the underlying distribution. In computer vision, where we have millions of training examples, this assumption is violated but the empirical performance may remain satisfactory.

Affine arithmetic-based B-spline surface intersection with GPU acceleration.

Because the B-spline surface intersection is a fundamental operation in geometric design software, it is important to make the surface intersection operation robust and efficient. As is well known, affine arithmetic is robust for calculating the surface intersection because it is able to not only find every branch of the intersection, but also deal with some singular cases, such as surface tangency. However, the classical affine arithmetic is defined only for the globally supported polynomials, and its computation is very time consuming, thus hampering its usefulness in practical applications, especially in geometric design.