Casual 6-DoF: Free-Viewpoint Panorama Using a Handheld 360° Camera.

Six degrees-of-freedom (6-DoF) video provides telepresence by enabling users to move around in the captured scene with a wide field of regard. Compared to methods requiring sophisticated camera setups, the image-based rendering method based on photogrammetry can work with images captured with any poses, which is more suitable for casual users. However, existing image-based rendering methods are based on perspective images.

Adaptive Light Estimation using Dynamic Filtering for Diverse Lighting Conditions.

High dynamic range (HDR) panoramic environment maps are widely used to illuminate virtual objects to blend with real-world scenes. However, in common applications for augmented and mixed-reality (AR/MR), capturing 360° surroundings to obtain an HDR environment map is often not possible using consumer-level devices. We present a novel light estimation method to predict 360° HDR environment maps from a single photograph with a limited field-of-view (FOV).

Reconstructing Reflection Maps Using a Stacked-CNN for Mixed Reality Rendering.

Corresponding lighting and reflectance between real and virtual objects is important for spatial presence in augmented and mixed reality (AR and MR) applications. We present a method to reconstruct real-world environmental lighting, encoded as a reflection map (RM), from a conventional photograph. To achieve this, we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360 RMs with varying roughness from a limited field of view, low dynamic range photograph.

Augmented Virtual Teleportation for High-Fidelity Telecollaboration.

Telecollaboration involves the teleportation of a remote collaborator to another real-world environment where their partner is located. The fidelity of the environment plays an important role for allowing corresponding spatial references in remote collaboration. We present a novel asymmetric platform, Augmented Virtual Teleportation (AVT), which provides high-fidelity telepresence of a remote VR user (VR-Traveler) into a real-world collaboration space to interact with a local AR user (AR-Host).

Blending Face Details: Synthesizing a Face Using Multiscale Face Models.

Creating realistic 3D face models is a challenging problem in computer graphics because humans are so sensitive to facial abnormalities. The authors propose a method to synthesize a 3D face model using weighted blending of multiscale details from different face models. Using multiscale continuous displacement maps (CDMs), they achieve full correspondences across multiple scales in the parameter space.

Low-Rank Matrix Completion to Reconstruct Incomplete Rendering Images.

Path tracing provides photo-realistic rendering in many applications but intermediate previsualization often suffers from distracting noise. Since the fundamental underlying problem is insufficient samples, we exploit the coherence of the visual signal to reconstruct missing samples, using a low-rank matrix completion framework. We present novel methods to construct low rank matrices for incomplete images including missing pixel, missing sub-pixel, and multi-frame scenarios.

MR360: Mixed Reality Rendering for 360° Panoramic Videos.

This paper presents a novel immersive system called MR360 that provides interactive mixed reality (MR) experiences using a conventional low dynamic range (LDR) 360° panoramic video (360-video) shown in head mounted displays (HMDs). MR360 seamlessly composites 3D virtual objects into a live 360-video using the input panoramic video as the lighting source to illuminate the virtual objects. Image based lighting (IBL) is perceptually optimized to provide fast and believable results using the LDR 360-video as the lighting source.

Reducing Visual Discomfort with HMDs Using Dynamic Depth of Field.

Although head-mounted displays (HMDs) are ideal devices for personal viewing of immersive stereoscopic content, exposure to VR applications on them results in significant discomfort for the majority of people, with symptoms including eye fatigue, headaches, nausea, and sweating. A conflict between accommodation and vergence depth cues on stereoscopic displays is a significant cause of visual discomfort. This article describes the results of an evaluation used to judge the effectiveness of dynamic depth-of-field (DoF) blur in an effort to reduce discomfort caused by exposure to stereoscopic content on HMDs.

Simulating liver deformation during respiration using sparse local features.

A new method deforms a 3D liver mesh in an arbitrary phase of respiration. During preprocessing, the method step defines a patient-specific deformation space using two polar shapes of the liver during respiration. 3D magnetic resonance imaging captures patient livers during exhaling and inhaling.

Scan-based volume animation driven by locally adaptive articulated registrations.

This paper describes a complete system to create anatomically accurate example-based volume deformation and animation of articulated body regions, starting from multiple in vivo volume scans of a specific individual. In order to solve the correspondence problem across volume scans, a template volume is registered to each sample. The wide range of pose variations is first approximated by volume blend deformation (VBD), providing proper initialization of the articulated subject in different poses.

Creating an animatable 3D volume hand model from in vivo MRI.

Volume graphics has obvious benefits to medical visualization, since it represents the complete 3D information of both surface appearance and the underlying anatomical structures. This study presents an approach to rapidly creating an animatable 3D volume from in vivo human hand MRI scans. The result is a fully articulated hand volume driven by intuitive joint control that respects rigid deformation of the bone structures and produces smooth deformations of both the skin surface and the interior soft tissue regions.