FingerSight: A Vibrotactile Wearable Ring for Assistance With Locating and Reaching Objects in Peripersonal Space.

This paper describes a prototype guidance system, "FingerSight," to help people without vision locate and reach to objects in peripersonal space. It consists of four evenly spaced tactors embedded into a ring worn on the index finger, with a small camera mounted on top. Computer-vision analysis of the camera image controls vibrotactile feedback, leading users to move their hand to near targets.

Integrating images from a moveable tracked display of three-dimensional data.

This paper describes a novel method for displaying data obtained by three-dimensional medical imaging, by which the position and orientation of a freely movable screen are optically tracked and used in real time to select the current slice from the data set for presentation. With this method, which we call a "freely moving medical image", the screen and imaged data are registered to a common coordinate system in space external to the user, at adjustable scale, and are available for free exploration. The three-dimensional image data occupy empty space, as if an invisible patient is being sliced by the moving screen.

In-Situ Force Augmentation Improves Surface Contact and Force Control.

Surgeons routinely perform surgery with noisy, sub-threshold, or obscured visual and haptic feedback, either due to the necessary surgical approach, or because the systems on which they are operating are exceedingly delicate. Technological solutions incorporating haptic feedback augmentation have been proposed to address these difficulties, but the consequences for motor control have not been directly investigated and quantified. In this paper, we present two isometric force generation tasks performed with a hand-held robotic tool that provides in-situ augmentation of force sensation.

Psychophysical evaluation of haptic perception under augmentation by a handheld device.

Objective: This study investigated the effectiveness of force augmentation in haptic perception tasks.

Background: Considerable engineering effort has been devoted to developing force augmented reality (AR) systems to assist users in delicate procedures like microsurgery. In contrast, far less has been done to characterize the behavioral outcomes of these systems, and no research has systematically examined the impact of sensory and perceptual processes on force augmentation effectiveness.

Refocusing a scanned laser projector for small and bright images: simultaneously controlling the profile of the laser beam and the boundary of the image.

This paper describes a projection system for augmenting a scanned laser projector to create very small, very bright images for use in a microsurgical augmented reality system. Normal optical design approaches are insufficient because the laser beam profile differs optically from the aggregate image. We propose a novel arrangement of two lens groups working together to simultaneously adjust both the laser beam of the projector (individual pixels) and the spatial envelope containing them (the entire image) to the desired sizes.

FingerSight: Fingertip Haptic Sensing of the Visual Environment.

We present a novel device mounted on the fingertip for acquiring and transmitting visual information through haptic channels. In contrast to previous systems in which the user interrogates an intermediate representation of visual information, such as a tactile display representing a camera generated image, our device uses a fingertip-mounted camera and haptic stimulator to allow the user to feel visual features directly from the environment. Visual features ranging from simple intensity or oriented edges to more complex information identified automatically about objects in the environment may be translated in this manner into haptic stimulation of the finger.

A model of motor performance during surface penetration: from physics to voluntary control.

The act of puncturing a surface with a hand-held tool is a ubiquitous but complex motor behavior that requires precise force control to avoid potentially severe consequences. We present a detailed model of puncture over a time course of approximately 1,000 ms, which is fit to kinematic data from individual punctures, obtained via a simulation with high-fidelity force feedback. The model describes puncture as proceeding from purely physically determined interactions between the surface and tool, through decline of force due to biomechanical viscosity, to cortically mediated voluntary control.

Pulmonary artery conduit in vivo dimensional requirements in a growing ovine model: comparisons with the ascending aorta.

Background And Aim Of The Study: The pulmonary trunk (PT) structure and function are abnormal in multiple congenital cardiovascular diseases. Existing surgical treatments of congenital malformations of the right ventricular outflow tract and PT do not provide a long-term replacement that can adapt to normal growth. Although there is strong interest in developing tissue-engineered approaches for PT conduit replacement, there remains an absence of any complete investigation of the native geometric growth patterns of the PT to serve as a necessary benchmark.

Mental visualization of objects from cross-sectional images.

We extended the classic anorthoscopic viewing procedure to test a model of visualization of 3D structures from 2D cross-sections. Four experiments were conducted to examine key processes described in the model, localizing cross-sections within a common frame of reference and spatiotemporal integration of cross sections into a hierarchical object representation. Participants used a hand-held device to reveal a hidden object as a sequence of cross-sectional images.

Real-time tomographic reflection in facilitating percutaneous access to the renal collecting system.

Background And Purpose: Real-time tomographic reflection is a novel technique that uses a geometrically fixed arrangement of a conventional ultrasound transducer, a transducer-incorporated monitor, and a half-silvered mirror. This device, dubbed the Sonic Flashlight, generates a virtual anatomically scaled image, obviating the need for a separate monitor. It may therefore facilitate invasive procedures, such as percutaneous access to the kidney.

The disembodied eye: consequences of displacing perception from action.

In our research, people use actions to expose hidden targets as planar images displayed either in situ or ex situ (displaced remotely). We show that because ex situ viewing impedes relating actions to their perceptual consequences, it impairs localizing targets, including compensating for surface deformation, and directing movement toward them. Using a 3D analogue of anorthoscopic perception, we demonstrate that spatio-temporal integration of contiguous planar slices is possible when action and perception are co-located, but not when they are separated.

Shells and Spheres: An n-Dimensional Framework for Medial-Based Image Segmentation.

We have developed a method for extracting anatomical shape models from n-dimensional images using an image analysis framework we call Shells and Spheres. This framework utilizes a set of spherical operators centered at each image pixel, grown to reach, but not cross, the nearest object boundary by incorporating "shells" of pixel intensity values while analyzing intensity mean, variance, and first-order moment. Pairs of spheres on opposite sides of putative boundaries are then analyzed to determine boundary reflectance which is used to further constrain sphere size, establishing a consensus as to boundary location.

Real-time tomographic holography for augmented reality.

The concept and instantiation of real-time tomographic holography (RTTH) for augmented reality is presented. RTTH enables natural hand-eye coordination to guide invasive medical procedures without requiring tracking or a head-mounted device. It places a real-time virtual image of an object's cross section into its actual location, without noticeable viewpoint dependence (e.

Visualizing 3D objects from 2D cross sectional images displayed in-situ versus ex-situ.

The present research investigates how mental visualization of a 3D object from 2D cross sectional images is influenced by displacing the images from the source object, as is customary in medical imaging. Three experiments were conducted to assess people's ability to integrate spatial information over a series of cross sectional images in order to visualize an object posed in 3D space. Participants used a hand-held tool to reveal a virtual rod as a sequence of cross-sectional images, which were displayed either directly in the space of exploration (in-situ) or displaced to a remote screen (ex-situ).

Peripherally inserted central catheter placement using the Sonic Flashlight.

The Sonic Flashlight is an ultrasound (US) device that projects real-time US images into patients with use of a semireflective/transparent mirror. The present study evaluated the feasibility of use of the Sonic Flashlight for clinical peripherally inserted central catheter placements, originally with the mirror located inside a sterile cover (n = 15), then with the mirror outside (n = 11). Successful access was obtained in all cases.

Peripherally inserted central catheter placement with the sonic flashlight: initial clinical trial by nurses.

Objective: We describe a case series constituting the first clinical trial by intravenous (IV) team nurses using the sonic flashlight (SF) for ultrasound guidance of peripherally inserted central catheter (PICC) placement.

Methods: Two IV team nurses with more than 10 years of experience with placing PICCs and 3 to 6 years of experience with ultrasound attempted to place PICCs under ultrasound guidance in patients requiring long-term IV access. One of two methods of ultrasound guidance was used: conventional ultrasound (CUS; 60 patients) or a new device called the SF (44 patients).

Fully automated common carotid artery and internal jugular vein identification and tracking using B-mode ultrasound.

We describe a fully automated ultrasound analysis system that tracks and identifies the common carotid artery (CCA) and the internal jugular vein (IJV). Our goal is to prevent inadvertent damage to the CCA when targeting the IJV for catheterization. The automated system starts by identifying and fitting ellipses to all the regions that look like major arteries or veins throughout each B-mode ultrasound image frame.

Learning to Reach to Locations Encoded from Imaging Displays.

The present study investigated how people learn to correct errors in actions directed toward cognitively encoded spatial locations. Subjects inserted a stylus to reach a hidden target localized by means of ultrasound imaging and portrayed with a scaled graph. As was found previously (Wu et al.

Amygdalae morphometry in late-life depression.

Objective: The amygdalae have been a focus of mood disorder research due to their key role in processing emotional information. It has been long known that depressed individuals demonstrate impaired functional performance while engaged in emotional tasks. The structural basis for these functional differences has been investigated via volumetric analysis with mixed findings.

Spatial Representations From Perception and Cognitive Mediation: The Case of Ultrasound.

Spatial representations can be derived not only by direct perception, but also through cognitive mediation. Conventional or ex-situ ultrasound displays, which displace imaged data to a remote screen, require both types of process. To determine the depth of a target hidden beneath a surface, ultrasound users must both perceive how deeply the ultrasound transducer indents the surface and interpret the on-screen image to visualize how deeply the target lies below the transducer.

Mental concatenation of perceptually and cognitively specified depth to represent locations in near space.

The purpose of this study was to examine how discrete segments of contiguous space arising from perceptual or cognitive channels are mentally concatenated. We induced and measured errors in each channel separately, then summed the psychophysical functions to accurately predict pointing to a depth specified by both together. In Experiment 1, subjects drew a line to match the visible indentation of a probe into a compressible surface.

Carotid artery and jugular vein tracking and differentiation using spatiotemporal analysis.

We have derived and evaluated parameters from ultrasound images of the neck to permit a computer to automatically characterize and differentiate between the carotid artery and jugular vein at image acquisition time during vascular interventions, given manually placed seed points. Our goal is to prevent inadvertent damage to the carotid artery when targeting the jugular vein for catheterization. We used a portable 10 MHz ultrasound system to acquire cross sectional B-mode ultrasound images of these great vessels at 10 fps.

Vascular access: comparison of US guidance with the sonic flashlight and conventional US in phantoms.

Purpose: To prospectively evaluate whether ultrasonography (US)-guided vascular access can be learned and performed faster with the sonic flashlight than with conventional US and to demonstrate sonic flashlight-guided vascular access in a cadaver.

Materials And Methods: Institutional review board approval and oral and written informed consent were obtained. The sonic flashlight replaces the standard US monitor with a real-time US image that appears to float beneath the skin and is displayed where it is scanned.

Laser needle guide for the sonic flashlight.

We have extended the real-time tomographic reflection display of the Sonic Flashlight to a laser guidance system that aims to improve safety and accuracy of needle insertion, especially for deep procedures. This guidance system is fundamentally different from others currently available. Two low-intensity lasers are mounted on opposite sides of a needle aimed parallel to the needle.

Optical merger of direct vision with virtual images for scaled teleoperation.

Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time.