Convolutional neural network for improved event-based Shack-Hartmann wavefront reconstruction.

Shack-Hartmann wavefront sensing is a technique for measuring wavefront aberrations, whose use in adaptive optics relies on fast position tracking of an array of spots. These sensors conventionally use frame-based cameras operating at a fixed sampling rate to report pixel intensities, even though only a fraction of the pixels have signal. Prior in-lab experiments have shown feasibility of event-based cameras for Shack-Hartmann wavefront sensing (SHWFS), asynchronously reporting the spot locations as log intensity changes at a microsecond time scale.

Continuously variable Fourier filters fabricated using varying angle glancing angle deposition for chip-scale spectroscopy.

Low-cost spectroscopy has received a great deal of attention in recent years in applications such as food inspection, disease detection, and manufacturing. Current spectroscopic systems rely on multiple optical components, making them mechanically fragile systems. In our previous work, we demonstrated the use of Fourier filtering using thin dielectric films.

Framework for improving DoLP and AoLP reconstruction quality in microgrid polarimeters.

In polarimetric imaging, degree and angle of linear polarization (DoLP and AoLP, respectively) are computed from ratios of Stokes parameters. In snapshot imagers, DoLP and AoLP are degraded by inherent mismatches between the spatial bandwidth of the S, S, and S parameters reconstructed by demosaicking from microgrid polarizer array (MPA)-sampled data. To overcome this, we rigorously show that log-MPA-sampled data approximately decouples DoLP and AoLP from the intensity component (S) in the spatial Fourier domain.

Time-Ordered Recent Event (TORE) Volumes for Event Cameras.

Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Unfortunately, most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. Many state-of-the-art algorithms for event cameras rely on interpolated event representations-obscuring crucial timing information, increasing the data volume, and limiting overall network performance.

Lossless White Balance for Improved Lossless CFA Image and Video Compression.

Color filter array is a spatial multiplexing of pixel-sized filters fabricated over pixel sensors in most color image sensors. The state-of-the-art lossless coding techniques of raw sensor data captured by such sensors leverage spatial or cross-color correlation using lifting schemes. In this paper, we propose a lifting-based lossless white balance algorithm.

Tunable optical filter using phase change materials for smart IR night vision applications.

In this paper we present a tunable filter using GeSbSeTe (GSST) phase change material. The design principle of the filter is based on a metal-insulator-metal (MIM) cavity operating in the reflection mode. This is intended for night vision applications that utilize 850nm as the illumination source.

Multi-Resolution Aitchison Geometry Image Denoising for Low-Light Photography.

In the low-photon imaging regime, noise in the image sensors is dominated by shot noise, best modeled statistically as Poisson distribution. In this work, we show that the Poisson likelihood function is very well matched with the Bayesian estimation of the "difference of log of contrast of pixel intensities." More specifically, our work is rooted in statistical compositional data analysis, whereby we reinterpret the Aitchison geometry as a multi-resolution analysis in the log-pixel domain.

L's approach: illumination pattern estimation technique for optical microscopy.

We propose a novel solution to the correction of illumination nonuniformity without removing the imaging sample. Calibration of the spatial illumination pattern in reflectance microscopy is challenging due to the fact that the illumination source is colocated with the objective lens and therefore cannot be observed directly. Our proposed methodology overcomes this by collecting three spatially translated images in a strategic way.

Distance Surface for Event-Based Optical Flow.

We propose DistSurf-OF, a novel optical flow method for neuromorphic cameras. Neuromorphic cameras (or event detection cameras) are an emerging sensor modality that makes use of dynamic vision sensors (DVS) to report asynchronously the log-intensity changes (called "events") exceeding a predefined threshold at each pixel. In absence of the intensity value at each pixel location, we introduce a notion of "distance surface"-the distance transform computed from the detected events-as a proxy for object texture.

Corrupted Reference Image Quality Assessment of Denoised Images.

We propose corrupted reference image quality assessment (CRIQA), a novel foundation for reasoning about image quality and image denoising problems jointly. In order to assess the visual quality of a processed image relative to an ideal reference image (not provided), we predict the full-reference image quality assessment (FRIQA) scores of denoised images without having the direct access to the ideal reference image, but with the help of the observed corrupted image, instead. Our simulation studies verify that the CRIQA scores of denoised images indeed agree with the corresponding FRIQA scores, and human subject studies confirm that CRIQA scores are more consistent with the perceived image denoising quality than the NRIQA scores.

Monte-Carlo Acceleration of Bilateral Filter and Non-Local Means.

We propose stochastic bilateral filter (SBF) and stochastic non-local means (SNLM), efficient randomized processes that agree with conventional bilateral filter (BF) and non-local means (NLM) on average, respectively. By Monte-Carlo, we repeat this process a few times with different random instantiations so that they can be averaged to attain the correct BF/NLM output. The computational bottleneck of the SBF and SNLM are constant with respect to the window size and the color dimension of the edge image, meaning the execution times for color and hyperspectral images are nearly the same as for the grayscale images.

Towards Optimal Denoising of Image Contrast.

Most conventional imaging modalities detect light indirectly by observing high-energy photons. The random nature of photon emission and detection is often the dominant sources of noise in imaging. Such case is referred to as photon-limited imaging, and the noise distribution is well modeled as Poisson.

Camera-Aware Multi-Resolution Analysis for Raw Image Sensor Data Compression.

We propose novel lossless and lossy compression schemes for color filter array (CFA) sampled images based on the Camera-A ware ulti-esolution nalysis, or CAMRA. Specifically, by CAMRA we refer to modifications that we make to wavelet transform of CFA sampled images in order to achieve a very high degree of decorrelation at the finest scale wavelet coefficients; and a series of color processing steps applied to the coarse scale wavelet coefficients, aimed at limiting the propagation of lossy compression errors through the subsequent camera processing pipeline. We validated our theoretical analysis and the performance of the proposed compression schemes using the images of natural scenes captured in a raw format.

Pixel Binning for High Dynamic Range Color Image Sensor Using Square Sampling Lattice.

We propose a new pixel binning scheme for color image sensors. We minimized distortion caused by binning by requiring that the superpixels lie on a square sampling lattice. The proposed binning schemes achieve the equivalent of 4.

Improved Denoising via Poisson Mixture Modeling of Image Sensor Noise.

This paper describes a study aimed at comparing the real image sensor noise distribution to the models of noise often assumed in image denoising designs. A quantile analysis in pixel, wavelet transform, and variance stabilization domains reveal that the tails of Poisson, signal-dependent Gaussian, and Poisson-Gaussian models are too short to capture real sensor noise behavior. A new Poisson mixture noise model is proposed to correct the mismatch of tail behavior.

Joint Defogging and Demosaicking.

Image defogging is a technique used extensively for enhancing visual quality of images in bad weather conditions. Even though defogging algorithms have been well studied, defogging performance is degraded by demosaicking artifacts and sensor noise amplification in distant scenes. In order to improve the visual quality of restored images, we propose a novel approach to perform defogging and demosaicking simultaneously.

Blind Deblurring and Denoising of Images Corrupted by Unidirectional Object Motion Blur and Sensor Noise.

Low light photography suffers from blur and noise. In this paper, we propose a novel method to recover a dense estimate of spatially varying blur kernel as well as a denoised and deblurred image from a single noisy and object motion blurred image. A proposed method takes the advantage of the sparse representation of double discrete wavelet transform-a generative model of image blur that simplifies the wavelet analysis of a blurred image-and the Bayesian perspective of modeling the prior distribution of the latent sharp wavelet coefficient and the likelihood function that makes the noise handling explicit.

Defocus Blur-Invariant Scale-Space Feature Extractions.

We propose modifications to scale-space feature extraction techniques scale-invariant feature transform (SIFT) and speeded up robust features (SURFs) that make the feature detection and description invariant to defocus blur. Specifically, the scale-space blob detection relies on the second derivative responses of images. Our analysis of circular defocus blur (which sufficiently approximates a real camera blur kernel) and its effect on scale-space blob detection suggests that fourth derivative-and not the usual second derivative-is optimal for detecting the blurred blobs, while multi-scale descriptors of blurred blobs are effective at establishing correspondences between the blurred images.

Fourier Spectral Filter Array for Optimal Multispectral Imaging.

Limitations to existing multispectral imaging modalities include speed, cost, range, spatial resolution, and application-specific system designs that lack versatility of the hyperspectral imaging modalities. In this paper, we propose a novel general-purpose single-shot passive multispectral imaging modality. Central to this design is a new type of spectral filter array (SFA) based not on the notion of spatially multiplexing narrowband filters, but instead aimed at enabling single-shot Fourier transform spectroscopy.

Fourier multispectral imaging.

Current multispectral imaging systems use narrowband filters to capture the spectral content of a scene, which necessitates different filters to be designed for each application. In this paper, we demonstrate the concept of Fourier multispectral imaging which uses filters with sinusoidally varying transmittance. We designed and built these filters employing a single-cavity resonance, and made spectral measurements with a multispectral LED array.

Minimum risk wavelet shrinkage operator for Poisson image denoising.

The pixel values of images taken by an image sensor are said to be corrupted by Poisson noise. To date, multiscale Poisson image denoising techniques have processed Haar frame and wavelet coefficients--the modeling of coefficients is enabled by the Skellam distribution analysis. We extend these results by solving for shrinkage operators for Skellam that minimizes the risk functional in the multiscale Poisson image denoising setting.

Camera processing with chromatic aberration.

Since the refractive index of materials commonly used for lens depends on the wavelengths of light, practical camera optics fail to converge light to a single point on an image plane. Known as chromatic aberration, this phenomenon distorts image details by introducing magnification error, defocus blur, and color fringes. Though achromatic and apochromatic lens designs reduce chromatic aberration to a degree, they are complex and expensive and they do not offer a perfect correction.

Noise parameter estimation for poisson corrupted images using variance stabilization transforms.

Noise is present in all images captured by real-world image sensors. Poisson distribution is said to model the stochastic nature of the photon arrival process and agrees with the distribution of measured pixel values. We propose a method for estimating unknown noise parameters from Poisson corrupted images using properties of variance stabilization.

Improved microgrid arrangement for integrated imaging polarimeters.

For almost 20 years, microgrid polarimetric imaging systems have been built using a 2×2 repeating pattern of polarization analyzers. In this Letter, we show that superior spatial resolution is achieved over this 2×2 case when the analyzers are arranged in a 2×4 repeating pattern. This unconventional result, in which a more distributed sampling pattern results in finer spatial resolution, is also achieved without affecting the conditioning of the polarimetric data-reduction matrix.

Color constancy with spatio-spectral statistics.

We introduce an efficient maximum likelihood approach for one part of the color constancy problem: removing from an image the color cast caused by the spectral distribution of the dominating scene illuminant. We do this by developing a statistical model for the spatial distribution of colors in white balanced images (i.e.