GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force.

Tactile sensing is an important perception mode for robots, but the existing tactile technologies have multiple limitations. What kind of tactile information robots need, and how to use the information, remain open questions. We believe a soft sensor surface and high-resolution sensing of geometry should be important components of a competent tactile sensor.

Visual wetness perception based on image color statistics.

Color vision provides humans and animals with the abilities to discriminate colors based on the wavelength composition of light and to determine the location and identity of objects of interest in cluttered scenes (e.g., ripe fruit among foliage).

Can you see what you feel? Color and folding properties affect visual-tactile material discrimination of fabrics.

Humans can often estimate tactile properties of objects from vision alone. For example, during online shopping, we can often infer material properties of clothing from images and judge how the material would feel against our skin. What visual information is important for tactile perception? Previous studies in material perception have focused on measuring surface appearance, such as gloss and roughness, and using verbal reports of material attributes and categories.

Accuracy and speed of material categorization in real-world images.

It is easy to visually distinguish a ceramic knife from one made of steel, a leather jacket from one made of denim, and a plush toy from one made of plastic. Most studies of material appearance have focused on the estimation of specific material properties such as albedo or surface gloss, and as a consequence, almost nothing is known about how we recognize material categories like leather or plastic. We have studied judgments of high-level material categories with a diverse set of real-world photographs, and we have shown (Sharan, 2009) that observers can categorize materials reliably and quickly.

Recognizing Materials using Perceptually Inspired Features.

Our world consists not only of objects and scenes but also of materials of various kinds. Being able to recognize the materials that surround us (e.g.

Image statistics for surface reflectance perception.

Human observers can distinguish the albedo of real-world surfaces even when the surfaces are viewed in isolation, contrary to the Gelb effect. We sought to measure this ability and to understand the cues that might underlie it. We took photographs of complex surfaces such as stucco and asked observers to judge their diffuse reflectance by comparing them to a physical Munsell scale.

Image statistics and the perception of surface qualities.

The world is full of surfaces, and by looking at them we can judge their material qualities. Properties such as colour or glossiness can help us decide whether a pancake is cooked, or a patch of pavement is icy. Most studies of surface appearance have emphasized textureless matte surfaces, but real-world surfaces, which may have gloss and complex mesostructure, are now receiving increased attention.

Recovering intrinsic images from a single image.

Interpreting real-world images requires the ability distinguish the different characteristics of the scene that lead to its final appearance. Two of the most important of these characteristics are the shading and reflectance of each point in the scene. We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image.

Straightness as a cue for luminance edge interpretation.

In order to determine the reflectance of a surface, it is necessary to discount luminance changes produced by illumination variation, a process that requires the visual system to respond differently to luminance changes that are due to illumination and reflectance. It is known that various cues can be used in this process. By measuring the strength of lightness illusions, we find evidence that straightness is, used as a cue: When a boundary is straight rather than curved, it has a greater tendency to be discounted, as if it were an illumination edge.

The geometry of the occluding contour and its effect on motion interpretation.

Form information related to occlusion is needed to correctly interpret image motion. This work describes one of a series of investigations into the form constraints on motion perception. In the present study, we focus specifically on the geometry of the occluding contour, and in particular on whether its influence on motion can be accounted for merely by its effect on perceived occlusion.

Statistical characterization of real-world illumination.

Although studies of vision and graphics often assume simple illumination models, real-world illumination is highly complex, with reflected light incident on a surface from almost every direction. One can capture the illumination from every direction at one point photographically using a spherical illumination map. This work illustrates, through analysis of photographically acquired, high dynamic range illumination maps, that real-world illumination possesses a high degree of statistical regularity.

Specular reflections and the perception of shape.

Many materials, including leaves, water, plastic, and chrome exhibit specular reflections. It seems reasonable that the visual system can somehow exploit specular reflections to recover three-dimensional (3D) shape. Previous studies (e.

Junctions and cost functions in motion interpretation.

Form, motion, occlusion, and perceptual organization are intimately related. We sought to assess the role of junctions in their interaction. We used stimuli based on a cross moving within an occluding aperture.

Real-world illumination and the perception of surface reflectance properties.

Under typical viewing conditions, we find it easy to distinguish between different materials, such as metal, plastic, and paper. Recognizing materials from their surface reflectance properties (such as lightness and gloss) is a nontrivial accomplishment because of confounding effects of illumination. However, if subjects have tacit knowledge of the statistics of illumination encountered in the real world, then it is possible to reject unlikely image interpretations, and thus to estimate surface reflectance even when the precise illumination is unknown.

Motion illusions as optimal percepts.

The pattern of local image velocities on the retina encodes important environmental information. Although humans are generally able to extract this information, they can easily be deceived into seeing incorrect velocities. We show that these 'illusions' arise naturally in a system that attempts to estimate local image velocity.