Phase contrast time-lapse microscopy datasets with automated and manual cell tracking annotations.

Phase contrast time-lapse microscopy is a non-destructive technique that generates large volumes of image-based information to quantify the behaviour of individual cells or cell populations. To guide the development of algorithms for computer-aided cell tracking and analysis, 48 time-lapse image sequences, each spanning approximately 3.5 days, were generated with accompanying ground truths for C2C12 myoblast cells cultured under 4 different media conditions, including with fibroblast growth factor 2 (FGF2), bone morphogenetic protein 2 (BMP2), FGF2 + BMP2, and control (no growth factor).

Panoptic Studio: A Massively Multiview System for Social Interaction Capture.

We present an approach to capture the 3D motion of a group of people engaged in a social interaction. The core challenges in capturing social interactions are: (1) occlusion is functional and frequent; (2) subtle motion needs to be measured over a space large enough to host a social group; (3) human appearance and configuration variation is immense; and (4) attaching markers to the body may prime the nature of interactions. The Panoptic Studio is a system organized around the thesis that social interactions should be measured through the integration of perceptual analyses over a large variety of view points.

Dense 3D Face Alignment from 2D Video for Real-Time Use.

To enable real-time, person-independent 3D registration from 2D video, we developed a 3D cascade regression approach in which facial landmarks remain invariant across pose over a range of approximately 60 degrees. From a single 2D image of a person's face, a dense 3D shape is registered in real time for each frame. The algorithm utilizes a fast cascade regression framework trained on high-resolution 3D face-scans of posed and spontaneous emotion expression.

Dense 3D Face Alignment from 2D Videos in Real-Time.

To enable real-time, person-independent 3D registration from 2D video, we developed a 3D cascade regression approach in which facial landmarks remain invariant across pose over a range of approximately 60 degrees. From a single 2D image of a person's face, a dense 3D shape is registered in real time for each frame. The algorithm utilizes a fast cascade regression framework trained on high-resolution 3D face-scans of posed and spontaneous emotion expression.

Interactive Cell Segmentation Based on Active and Semi-Supervised Learning.

Automatic cell segmentation can hardly be flawless due to the complexity of image data particularly when time-lapse experiments last for a long time without biomarkers. To address this issue, we propose an interactive cell segmentation method by classifying feature-homogeneous superpixels into specific classes, which is guided by human interventions. Specifically, we propose to actively select the most informative superpixels by minimizing the expected prediction error which is upper bounded by the transductive Rademacher complexity, and then query for human annotations.

Data-Driven Objectness.

We propose a data-driven approach to estimate the likelihood that an image segment corresponds to a scene object (its "objectness") by comparing it to a large collection of example object regions. We demonstrate that when the application domain is known, for example, in our case activity of daily living (ADL), we can capture the regularity of the domain specific objects using millions of exemplar object regions. Our approach to estimating the objectness of an image region proceeds in two steps: 1) finding the exemplar regions that are the most similar to the input image segment; 2) calculating the objectness of the image segment by combining segment properties, mutual consistency across the nearest exemplar regions, and the prior probability of each exemplar region.

Discriminative data transform for image feature extraction and classification.

Good feature design is important to achieve effective image classification. This paper presents a novel feature design with two main contributions. First, prior to computing the feature descriptors, we propose to transform the images with learning-based filters to obtain more representative feature descriptors.

Apoptosis detection for non-adherent cells in time-lapse phase contrast microscopy.

This paper proposes a vision-based method for detecting apoptosis (programmed cell death), which is essential for non-perturbative monitoring of cell expansion. Our method targets non-adherent cells, which float or are suspended freely in the culture medium-in contrast to adherent cells, which are attached to a petri dish. The method first detects cell regions and tracks them over time, resulting in the construction of cell tracklets.

Efficient phase contrast microscopy restoration applied for muscle myotube detection.

This paper proposes a new image restoration method for phase contrast microscopy as a mean to enhance the quality of images prior to image analysis. Compared to state-of-the-art image restoration algorithms, our method has a more solid theoretical foundation and is orders of magnitude more efficient in computation. We validated the proposed method by applying it to automated muscle myotube detection, a challenging problem that has not been tackled without staining images.

Spatio-temporal Event Classification using Time-series Kernel based Structured Sparsity.

In many behavioral domains, such as facial expression and gesture, sparse structure is prevalent. This sparsity would be well suited for event detection but for one problem. Features typically are confounded by alignment error in space and time.

Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features.

Phase-contrast microscopy is one of the most common and convenient imaging modalities to observe long-term multi-cellular processes, which generates images by the interference of lights passing through transparent specimens and background medium with different retarded phases. Despite many years of study, computer-aided phase contrast microscopy analysis on cell behavior is challenged by image qualities and artifacts caused by phase contrast optics. Addressing the unsolved challenges, the authors propose (1) a phase contrast microscopy image restoration method that produces phase retardation features, which are intrinsic features of phase contrast microscopy, and (2) a semi-supervised learning based algorithm for cell segmentation, which is a fundamental task for various cell behavior analysis.

Multi-label classification for the analysis of human motion quality.

Knowing how well an activity is performed is important for home rehabilitation. We would like to not only know if a motion is being performed correctly, but also in what way the motion is incorrect so that we may provide feedback to the user. This paper describes methods for assessing human motion quality using body-worn tri-axial accelerometers and gyroscopes.

Phase contrast image restoration via dictionary representation of diffraction patterns.

The restoration of microscopy images makes the segmentation and detection of cells easier and more reliable, which facilitates automated cell tracking and cell behavior analysis. In this paper, the authors analyze the image formation process of phase contrast images and propose an image restoration method based on the dictionary representation of diffraction patterns. By formulating and solving a min-l1 optimization problem, each pixel is restored into a feature vector corresponding to the dictionary representation.

Apoptosis detection for adherent cell populations in time-lapse phase-contrast microscopy images.

The detection of apoptosis, or programmed cell death, is important to understand the underlying mechanism of cell development. At present, apoptosis detection resorts to fluorescence or colorimetric assays, which may affect cell behavior and thus not allow long-term monitoring of intact cells. In this work, we present an image analysis method to detect apoptosis in time-lapse phase-contrast microscopy, which is nondestructive imaging.

Understanding the phase contrast optics to restore artifact-free microscopy images for segmentation.

Phase contrast, a noninvasive microscopy imaging technique, is widely used to capture time-lapse images to monitor the behavior of transparent cells without staining or altering them. Due to the optical principle, phase contrast microscopy images contain artifacts such as the halo and shade-off that hinder image segmentation, a critical step in automated microscopy image analysis. Rather than treating phase contrast microscopy images as general natural images and applying generic image processing techniques on them, we propose to study the optical properties of the phase contrast microscope to model its image formation process.

Mechanical characterization of adult stem cells from bone marrow and perivascular niches.

Therapies using adult stem cells often require mechanical manipulation such as injection or incorporation into scaffolds. However, force-induced rupture and mechanosensitivity of cells during manipulation is largely ignored. Here, we image cell mechanical structures and perform a biophysical characterization of three different types of human adult stem cells: bone marrow CD34+ hematopoietic, bone marrow mesenchymal and perivascular mesenchymal stem cells.

Automatic cell tracking applied to analysis of cell migration in wound healing assay.

The wound healing assay in vitro is widely used for research and discovery in biology and medicine. This assay allows for observing the healing process in vitro in which the cells on the edges of the artificial wound migrate toward the wound area. The influence of different culture conditions can be measured by observing the change in the size of the wound area.

Data-driven prediction of stem cell expansion cultures.

Stem cell expansion culture aims to generate sufficient number of clinical-grade cells for cell-based therapies. One challenge for ex vivo expansion is to decide the appropriate time to perform subculture. Traditionally, this decision has been reliant on human estimation of cell confluency and predicting when confluency will approach a desired threshold.

Tailoring the trajectory of cell rolling with cytotactic surfaces.

Cell separation technology is a key tool for biological studies and medical diagnostics that relies primarily on chemical labeling to identify particular phenotypes. An emergent method of sorting cells based on differential rolling on chemically patterned substrates holds potential benefits over existing technologies, but the underlying mechanisms being exploited are not well characterized. In order to better understand cell rolling on complex surfaces, a microfluidic device with chemically patterned stripes of the cell adhesion molecule P-selectin was designed.

An engineered approach to stem cell culture: automating the decision process for real-time adaptive subculture of stem cells.

Current cell culture practices are dependent upon human operators and remain laborious and highly subjective, resulting in large variations and inconsistent outcomes, especially when using visual assessments of cell confluency to determine the appropriate time to subculture cells. Although efforts to automate cell culture with robotic systems are underway, the majority of such systems still require human intervention to determine when to subculture. Thus, it is necessary to accurately and objectively determine the appropriate time for cell passaging.

A semi-Markov model for mitosis segmentation in time-lapse phase contrast microscopy image sequences of stem cell populations.

We propose a semi-Markov model trained in a max-margin learning framework for mitosis event segmentation in large-scale time-lapse phase contrast microscopy image sequences of stem cell populations. Our method consists of three steps. First, we apply a constrained optimization based microscopy image segmentation method that exploits phase contrast optics to extract candidate subsequences in the input image sequence that contains mitosis events.

Restoring DIC microscopy images from multiple shear directions.

Differential Interference Contrast (DIC) microscopy is a nondestructive imaging modality that has been widely used by biologists to capture microscopy images of live biological specimens. However, as a qualitative technique, DIC microscopy records specimen's physical properties in an indirect way by mapping the gradient of specimen's optical path length (OPL) into the image intensity. In this paper, we propose to restore DIC microscopy images by quantitatively estimating specimen's OPL from a collection of DIC images captured from multiple shear directions.

Multiaperture telecentric lens for 3D reconstruction.

We present a telecentric lens that is able to gain 3D information. The proposed lens system has multiple aperture stops, which enable it to capture multidirectional parallel light rays, while a conventional telecentric lens has only one aperture stop and can capture only light rays that are perpendicular to the lens. We explain the geometry of the multiaperture telecentric system and show that correspondences fall on a line like those in a conventional stereo.

Robustly Aligning a Shape Model and Its Application to Car Alignment of Unknown Pose.

Precisely localizing in an image a set of feature points that form a shape of an object, such as car or face, is called alignment. Previous shape alignment methods attempted to fit a whole shape model to the observed data, based on the assumption of Gaussian observation noise and the associated regularization process. However, such an approach, though able to deal with Gaussian noise in feature detection, turns out not to be robust or precise because it is vulnerable to gross feature detection errors or outliers resulting from partial occlusions or spurious features from the background or neighboring objects.

Automated mitosis detection of stem cell populations in phase-contrast microscopy images.

Due to the enormous potential and impact that stem cells may have on regenerative medicine, there has been a rapidly growing interest for tools to analyze and characterize the behaviors of these cells in vitro in an automated and high throughput fashion. Among these behaviors, mitosis, or cell division, is important since stem cells proliferate and renew themselves through mitosis. However, current automated systems for measuring cell proliferation often require destructive or sacrificial methods of cell manipulation such as cell lysis or in vitro staining.