Aligning Visual Prosthetic Development With Implantee Needs.

Purpose: Visual prosthetics are a promising assistive technology for vision loss, yet research often overlooks the human aspects of this technology. While previous studies focus on the perceptual experiences or attitudes of implant recipients (implantees), a systematic account of how current implants are being used in everyday life is still lacking.

Methods: We interviewed six recipients of the most widely used visual implants (Argus II and Orion) and six leading researchers in the field.

Phenotypic and genetic characteristics of retinal vascular parameters and their association with diseases.

Fundus images allow for non-invasive assessment of the retinal vasculature whose features provide important information on health. Using a fully automated image processing pipeline, we extract 17 different morphological vascular phenotypes, including median vessels diameter, diameter variability, main temporal angles, vascular density, central retinal equivalents, the number of bifurcations, and tortuosity, from over 130,000 fundus images of close to 72,000 UK Biobank subjects. We perform genome-wide association studies of these phenotypes.

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice.

Despite their immense success as a model of macaque visual cortex, deep convolutional neural networks (CNNs) have struggled to predict activity in visual cortex of the mouse, which is thought to be strongly dependent on the animal's behavioral state. Furthermore, most computational models focus on predicting neural responses to static images presented under head fixation, which are dramatically different from the dynamic, continuous visual stimuli that arise during movement in the real world. Consequently, it is still unknown how natural visual input and different behavioral variables may integrate over time to generate responses in primary visual cortex (V1).

Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses.

Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. Exact placement of implants and differences in individual perception lead to significant variations in stimulus response, making personalized stimulus optimization a key challenge. Bayesian optimization could be used to optimize patient-specific stimulation parameters with limited noisy observations, but is not feasible for high-dimensional stimuli.

Aligning visual prosthetic development with implantee needs.

Purpose: Visual prosthetics are a promising assistive technology for vision loss, yet research often overlooks the human aspects of this technology. While previous studies focus on the perceptual experiences or attitudes of implant recipients (implantees) , a systematic account of how current implants are being used in everyday life is still lacking.

Methods: We interviewed six recipients of the most widely used visual implants (Argus II and Orion) and six leading researchers in the field.

Stress affects navigation strategies in immersive virtual reality.

There are known individual differences in both the ability to learn the layout of novel environments and the flexibility of strategies for navigating known environments. However, it is unclear how navigational abilities are impacted by high-stress scenarios. Here we used immersive virtual reality (VR) to develop a novel behavioral paradigm to examine navigation under dynamically changing situations.

Axonal stimulation affects the linear summation of single-point perception in three Argus II users.

Retinal implants use electrical stimulation to elicit perceived flashes of light ('phosphenes'). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.

Explainable machine learning predictions of perceptual sensitivity for retinal prostheses.

Retinal prostheses evoke visual precepts by electrically stimulating functioning cells in the retina. Despite high variance in perceptual thresholds across subjects, among electrodes within a subject, and over time, retinal prosthesis users must undergo 'system fitting', a process performed to calibrate stimulation parameters according to the subject's perceptual thresholds. Although previous work has identified electrode-retina distance and impedance as key factors affecting thresholds, an accurate predictive model is still lacking.

Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses.

Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capabilities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here, we frame this as an end-to-end optimization problem, where a deep neural network stimulus encoder is trained to invert a known and fixed forward model that approximates the underlying biological system.

Axonal stimulation affects the linear summation of single-point perception in three Argus II users.

Purpose: Retinal implants use electrical stimulation to elicit perceived flashes of light ("phosphenes"). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice.

Despite their immense success as a model of macaque visual cortex, deep convolutional neural networks (CNNs) have struggled to predict activity in visual cortex of the mouse, which is thought to be strongly dependent on the animal's behavioral state. Furthermore, most computational models focus on predicting neural responses to static images presented under head fixation, which are dramatically different from the dynamic, continuous visual stimuli that arise during movement in the real world. Consequently, it is still unknown how natural visual input and different behavioral variables may integrate over time to generate responses in primary visual cortex (V1).

Retinal ganglion cells undergo cell type-specific functional changes in a computational model of cone-mediated retinal degeneration.

Introduction: Understanding the retina in health and disease is a key issue for neuroscience and neuroengineering applications such as retinal prostheses. During degeneration, the retinal network undergoes complex and multi-stage neuroanatomical alterations, which drastically impact the retinal ganglion cell (RGC) response and are of clinical importance. Here we present a biophysically detailed model of the cone pathway in the retina that simulates the network-level response to both light and electrical stimulation.

A systematic review of extended reality (XR) for understanding and augmenting vision loss.

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present.

Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition.

Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing.

Explainable Machine Learning Predictions of Perceptual Sensitivity for Retinal Prostheses.

To provide appropriate levels of stimulation, retinal prostheses must be calibrated to an individual's perceptual thresholds ('system fitting'), despite thresholds varying drastically across subjects, across electrodes within a subject, and over time. Although previous work has identified electrode-retina distance and impedance as key factors affecting thresholds, an accurate predictive model is still lacking. To address these challenges, we 1) fitted machine learning (ML) models to a large longitudinal dataset with the goal of predicting individual electrode thresholds and deactivation as a function of stimulus, electrode, and clinical parameters ('predictors') and 2) leveraged explainable artificial intelligence (XAI) to reveal which of these predictors were most important.

Retinal ganglion cells undergo cell typeâ€"specific functional changes in a biophysically detailed model of retinal degeneration.

Understanding the retina in health and disease is a key issue for neuroscience and neuroengineering applications such as retinal prostheses. During degeneration, the retinal network undergoes complex and multi-stage neuroanatomical alterations, which drastically impact the retinal ganglion cell (RGC) response and are of clinical importance. Here we present a biophysically detailed model of retinal degeneration that simulates the network-level response to both light and electrical stimulation as a function of disease progression.

Towards a: AI-powered artificial vision for the treatment of incurable blindness.

How can we return a functional form of sight to people who are living with incurable blindness? Despite recent advances in the development of visual neuroprostheses, the quality of current prosthetic vision is still rudimentary and does not differ much across different device technologies.Rather than aiming to represent the visual scene as naturally as possible, acould provide visual augmentations through the means of artificial intelligence-based scene understanding, tailored to specific real-world tasks that are known to affect the quality of life of people who are blind, such as face recognition, outdoor navigation, and self-care.Complementary to existing research aiming to restore natural vision, we propose a patient-centered approach to incorporate deep learning-based visual augmentations into the next generation of devices.

Factors affecting two-point discrimination in Argus II patients.

Two of the main obstacles to the development of epiretinal prosthesis technology are electrodes that require current amplitudes above safety limits to reliably elicit percepts, and a failure to consistently elicit pattern vision. Here, we explored the causes of high current amplitude thresholds and poor spatial resolution within the Argus II epiretinal implant. We measured current amplitude thresholds and two-point discrimination (the ability to determine whether one or two electrodes had been stimulated) in 3 blind participants implanted with Argus II devices.

Immersive Virtual Reality Simulations of Bionic Vision.

Bionic vision uses neuroprostheses to restore useful vision to people living with incurable blindness. However, a major outstanding challenge is predicting what people "see" when they use their devices. The limited field of view of current devices necessitates head movements to scan the scene, which is difficult to simulate on a computer screen.

Cortical Motion Perception Emerges from Dimensionality Reduction with Evolved Spike-Timing-Dependent Plasticity Rules.

The nervous system is under tight energy constraints and must represent information efficiently. This is particularly relevant in the dorsal part of the medial superior temporal area (MSTd) in primates where neurons encode complex motion patterns to support a variety of behaviors. A sparse decomposition model based on a dimensionality reduction principle known as non-negative matrix factorization (NMF) was previously shown to account for a wide range of monkey MSTd visual response properties.

Learning to see again: Perceptual learning of simulated abnormal on- off-cell population responses in sighted individuals.

Many forms of artificial sight recovery, such as electronic implants and optogenetic proteins, generally cause simultaneous, rather than complementary firing of on- and off-center retinal cells. Here, using virtual patients-sighted individuals viewing distorted input-we examine whether plasticity might compensate for abnormal neuronal population responses. Five participants were dichoptically presented with a combination of original and contrast-reversed images.

A Computational Model of Phosphene Appearance for Epiretinal Prostheses.

Retinal neuroprostheses are the only FDA-approved treatment option for blinding degenerative diseases. A major outstanding challenge is to develop a computational model that can accurately predict the elicited visual percepts (phosphenes) across a wide range of electrical stimuli. Here we present a phenomenological model that predicts phosphene appearance as a function of stimulus amplitude, frequency, and pulse duration.

Model-Based Recommendations for Optimal Surgical Placement of Epiretinal Implants.

A major limitation of current electronic retinal implants is that in addition to stimulating the intended retinal ganglion cells, they also stimulate passing axon fibers, producing perceptual 'streaks' that limit the quality of the generated visual experience. Recent evidence suggests a dependence between the shape of the elicited visual percept and the retinal location of the stimulating electrode. However, this knowledge has yet to be incorporated into the surgical placement of retinal implants.

Data-driven models in human neuroscience and neuroengineering.

Discoveries in modern human neuroscience are increasingly driven by quantitative understanding of complex data. Data-intensive approaches to modeling have promise to dramatically advance our understanding of the brain and critically enable neuroengineering capabilities. In this review, we provide an accessible primer to modern modeling approaches and highlight recent data-driven discoveries in the domains of neuroimaging, single-neuron and neuronal population responses, and device neuroengineering.