Learning generalizable behaviors from demonstration.

Generalizing prior experiences to complete new tasks is a challenging and unsolved problem in robotics. In this work, we explore a novel framework for control of complex systems called Primitive Imitation for Control (). The approach combines ideas from imitation learning, task decomposition, and novel task sequencing to generalize from demonstrations to new behaviors.

Eye Movement Control in the Argus II Retinal-Prosthesis Enables Reduced Head Movement and Better Localization Precision.

Purpose: Visual scanning by sighted individuals is done using eye and head movements. In contrast, scanning using the Argus II is solely done by head movement, since eye movements can introduce localization errors. Here, we tested if a scanning mode utilizing eye movements increases visual stability and reduces head movements in Argus II users.

Eye movements as a marker for visual prosthesis spatial mapping - A feasibility study using a blind patient implanted with the Argus II retinal prosthesis.

Spatial mapping, the location in space of a perceived location due to an implanted electrode's electrical stimulation is important in the design of visual prostheses. Generally, a visual prosthesis system consists of an implanted electrode array, an external camera that acquires the image, and a transmitter that sends the information to the implanted electrodes. In cortical visual implant, the layout of the implanted array in most cases does not match the retinotopic map and it is necessary to find the location of the percept of each electrode in world coordinates.

Embedded clutter reduction and face detection algorithms for a visual prosthesis.

Retinal prosthetic devices can significantly and positively impact the ability of visually challenged individuals to live a more independent life. We describe a visual processing system which leverages image analysis techniques to produce visual patterns and allows the user to more effectively perceive their environment. These patterns are used to stimulate a retinal prosthesis to allow self guidance and a higher degree of autonomy for the affected individual.

Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject.

Objective: We used native sensorimotor representations of fingers in a brain-machine interface (BMI) to achieve immediate online control of individual prosthetic fingers.

Approach: Using high gamma responses recorded with a high-density electrocorticography (ECoG) array, we rapidly mapped the functional anatomy of cued finger movements. We used these cortical maps to select ECoG electrodes for a hierarchical linear discriminant analysis classification scheme to predict: (1) if any finger was moving, and, if so, (2) which digit was moving.

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas.

Collaborative approach in the development of high-performance brain-computer interfaces for a neuroprosthetic arm: translation from animal models to human control.

Our research group recently demonstrated that a person with tetraplegia could use a brain-computer interface (BCI) to control a sophisticated anthropomorphic robotic arm with skill and speed approaching that of an able-bodied person. This multiyear study exemplifies important principles in translating research from foundational theory and animal experiments into a clinical study. We present a roadmap that may serve as an example for other areas of clinical device research as well as an update on study results.

Simultaneous neural control of simple reaching and grasping with the modular prosthetic limb using intracranial EEG.

Intracranial electroencephalographic (iEEG) signals from two human subjects were used to achieve simultaneous neural control of reaching and grasping movements with the Johns Hopkins University Applied Physics Lab (JHU/APL) Modular Prosthetic Limb (MPL), a dexterous robotic prosthetic arm. We performed functional mapping of high gamma activity while the subject made reaching and grasping movements to identify task-selective electrodes. Independent, online control of reaching and grasping was then achieved using high gamma activity from a small subset of electrodes with a model trained on short blocks of reaching and grasping with no further adaptation.

Behavioral demonstration of a somatosensory neuroprosthesis.

Tactile sensation is critical for effective object manipulation, but current prosthetic upper limbs make no provision for delivering somesthetic feedback to the user. For individuals who require use of prosthetic limbs, this lack of feedback transforms a mundane task into one that requires extreme concentration and effort. Although vibrotactile motors and sensory substitution devices can be used to convey gross sensations, a direct neural interface is required to provide detailed and intuitive sensory feedback.