A Multi-aperture Coaxial Projector Balancing Shadow Suppression and Deblurring.

This paper proposes a projection system that optically removes the cast shadow in projection mapping. Specifically, we realize the large-aperture (LA) projection using a large-format Fresnel lens to suppress cast shadows by condensing the projection light from a wide viewing angle. However, the resolution and contrast of the projected results are significantly degraded by defocus blur, veiling glare, and stray light caused by the aberration of an LA Fresnel lens.

Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors.

Projection mapping (PM) is a technique that enhances the appearance of real-world surfaces using projected images, enabling multiple people to view augmentations simultaneously, thereby facilitating communication and collaboration. However, PM typically requires a dark environment to achieve high-quality projections, limiting its practicality. In this paper, we overcome this limitation by replacing conventional room lighting with heterogeneous projectors.

Efficient Distortion-Free Neural Projector Deblurring in Dynamic Projection Mapping.

Dynamic Projection Mapping (DPM) necessitates geometric compensation of the projection image based on the position and orientation of moving objects. Additionally, the projector's shallow depth of field results in pronounced defocus blur even with minimal object movement. Achieving delay-free DPM with high image quality requires real-time implementation of geometric compensation and projector deblurring.

Neural Projection Mapping Using Reflectance Fields.

We introduce a high resolution spatially adaptive light source, or a projector, into a neural reflectance field that allows to both calibrate the projector and photo realistic light editing. The projected texture is fully differentiable with respect to all scene parameters, and can be optimized to yield a desired appearance suitable for applications in augmented reality and projection mapping. Our neural field consists of three neural networks, estimating geometry, material, and transmittance.

Shadowless Projection Mapping using Retrotransmissive Optics.

This paper presents a shadowless projection mapping system for interactive applications in which a target surface is frequently occluded from a projector with a user's body. We propose a delay-free optical solution for this critical problem. Specifically, as the primary technical contribution, we apply a large format retrotransmissive plate to project images onto the target surface from wide viewing angles.

A Monocular Projector-Camera System Using Modular Architecture.

This paper presents a monocular projector-camera (procam) system using modular architecture based on relay optics. Conventional coaxial procam systems cannot support (1) online changes to lens settings (zoom and focus) and (2) wide-angle projection mapping. We develop design guidelines for a proposed procam system that would solve these restrictions and address the proposed system's unique technical issue of crosstalk between the camera and projector pixels.

Online Projector Deblurring Using a Convolutional Neural Network.

Projector deblurring is an important technology for dynamic projection mapping (PM), where the distance between a projector and a projection surface changes in time. However, conventional projector deblurring techniques do not support dynamic PM because they need to project calibration patterns to estimate the amount of defocus blur each time the surface moves. We present a deep neural network that can compensate for defocus blur in dynamic PM.

HaptoMapping: Visuo-Haptic Augmented Reality by Embedding User-Imperceptible Tactile Display Control Signals in a Projected Image.

This article proposes HaptoMapping, a projection-based visuo-haptic augmented reality (VHAR) system, that can render visual and haptic content independently and present consistent visuo-haptic sensations on physical surfaces. HaptoMapping controls wearable haptic displays by embedded control signals that are imperceptible to the user in projected images using a pixel-level visible light communication technique. The prototype system is comprised of a high-speed projector and three types of haptic devices-finger worn, stylus, and arm mounted.

Directionally Decomposing Structured Light for Projector Calibration.

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF.

Multifocal Stereoscopic Projection Mapping.

Stereoscopic projection mapping (PM) allows a user to see a three-dimensional (3D) computer-generated (CG) object floating over physical surfaces of arbitrary shapes around us using projected imagery. However, the current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues, which causes a vergence-accommodation conflict (VAC). Therefore, we propose a multifocal approach to mitigate VAC in stereoscopic PM.

IlluminatedZoom: spatially varying magnified vision using periodically zooming eyeglasses and a high-speed projector.

Spatial zooming and magnification, which control the size of only a portion of a scene while maintaining its context, is an essential interaction technique in augmented reality (AR) systems. It has been applied in various AR applications including surgical navigation, visual search support, and human behavior control. However, spatial zooming has been implemented only on video see-through displays and not been supported by optical see-through displays.

Immersive Virtual Reality Reminiscence Reduces Anxiety in the Oldest-Old Without Causing Serious Side Effects: A Single-Center, Pilot, and Randomized Crossover Study.

Dementia is one the major problems of aging societies, and, novel and effective non-drug therapies are required as interventions in the oldest-old to prevent cognitive decline. This study aims to examine the efficacy and safety of reminiscence using immersive virtual reality (iVR reminiscence) focusing on anxiety that often appears with cognitive decline. The secondary objective is to reveal the preference for VR image types for reminiscence: live-action (LA) or computer graphics (CG).

ProDebNet: projector deblurring using a convolutional neural network.

Projection blur can occur in practical use cases that have non-planar and/or multi-projection display surfaces with various scattering characteristics because the surface often causes defocus and subsurface scattering. To address this issue, we propose ProDebNet, an end-to-end real-time projection deblurring network that synthesizes a projection image to minimize projection blur. The proposed method generates a projection image without explicitly estimating any geometry or scattering characteristics of the projection screen, which makes real-time processing possible.

Illuminated Focus: Vision Augmentation using Spatial Defocusing via Focal Sweep Eyeglasses and High-Speed Projector.

Aiming at realizing novel vision augmentation experiences, this paper proposes the IlluminatedFocus technique, which spatially defocuses real-world appearances regardless of the distance from the user's eyes to observed real objects. With the proposed technique, a part of a real object in an image appears blurred, while the fine details of the other part at the same distance remain visible. We apply Electrically Focus-Tunable Lenses (ETL) as eyeglasses and a synchronized high-speed projector as illumination for a real scene.

FibAR: Embedding Optical Fibers in 3D Printed Objects for Active Markers in Dynamic Projection Mapping.

This paper presents a novel active marker for dynamic projection mapping (PM) that emits a temporal blinking pattern of infrared (IR) light representing its ID. We used a multi-material three dimensional (3D) printer to fabricate a projection object with optical fibers that can guide IR light from LEDs attached on the bottom of the object. The aperture of an optical fiber is typically very small; thus, it is unnoticeable to human observers under projection and can be placed on a strongly curved part of a projection surface.

Computational Phase-Modulated Eyeglasses.

We present computational phase-modulated eyeglasses, a see-through optical system that modulates the view of the user using phase-only spatial light modulators (PSLM). A PSLM is a programmable reflective device that can selectively retardate, or delay, the incoming light rays. As a result, a PSLM works as a computational dynamic lens device.

Speeded-Up Focus Control of Electrically Tunable Lens by Sparse Optimization.

Electrically tunable lenses (ETL), also known as liquid lenses, can be focused at various distances by changing the electric signal applied on the lens. ETLs require no mechanical structures, and therefore, provide a more compact and inexpensive focus control than conventional computerized translation stages. They have been exploited in a wide range of imaging and display systems and enabled novel applications for the last several years.

Light Attenuation Display: Subtractive See-Through Near-Eye Display via Spatial Color Filtering.

We present a display for optical see-through near-eye displays based on light attenuation, a new paradigm that forms images by spatially subtracting colors of light. Existing optical see-through head-mounted displays (OST-HMDs) form virtual images in an additive manner-they optically combine the light from an embedded light source such as a microdisplay into the users' field of view (FoV). Instead, our light attenuation display filters the color of the real background light pixel-wise in the users' see-through view, resulting in an image as a spatial color filter.

Seamless Multi-Projection Revisited.

This paper introduces a novel photometric compensation technique for inter-projector luminance and chrominance variations. Although it sounds as a classical technical issue, to the best of our knowledge there is no existing solution to alleviate the spatial non-uniformity among strongly heterogeneous projectors at perceptually acceptable quality. Primary goal of our method is increasing the perceived seamlessness of the projection system by automatically generating an improved and consistent visual quality.

FleXeen: Visually Manipulating Perceived Fabric Bending Stiffness in Spatial Augmented Reality.

The appearance of fabric motion is suggested to affect the human perception of bending stiffness. This study presents a novel spatial augmented reality, or projection mapping, approach that can visually manipulate the perceived bending stiffness of a fabric. Particularly, we proposed a flow enhancement method that can change the apparent fabric motion by using a simple optical flow analysis technique rather than complex physical simulations for interactive applications.

Simple rules for construction of a geometric nest structure by pufferfish.

A small (~10 cm) male pufferfish (Torquigener albomaculosus) builds a large (~2 m) sandy nest structure, resembling a mysterious crop circle, to attract females. The circle consists of radially arranged deep ditches in the outer ring region, and maze-like shallow ditches in the central region. The configuration is geometrical.

Non-Contact Thermo-Visual Augmentation by IR-RGB Projection.

This paper presents an approach for non-contact haptic augmentation with spatial augmented reality (SAR). We construct a thermo-visual projection system by combining a standard RGB projector and a fabricated infrared (IR) projector. The primary contribution of this paper is that we conduct thorough psychophysical experiments to investigate a design guideline for spatiotemporal projection patterns for both RGB and IR projectors to render a warm object with high presence.

HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection.

Spatial augmented reality (SAR) pursues realism in rendering materials and objects. To advance this goal, we propose a hybrid SAR (HySAR) that combines a projector with optical see-through head-mounted displays (OST-HMD). In an ordinary SAR scenario with co-located viewers, the viewers perceive the same virtual material on physical surfaces.