PDM-ENLOR for segmentation of mouse brain gene expression images.

Statistical shape models, such as Active Shape Models (ASMs), suffer from their inability to represent a large range of variations of a complex shape and to account for the large errors in detection of (point) landmarks. We propose a method, PDM-ENLOR (Point Distribution Model-based ENsemble of LOcal Regressors), that overcomes these limitations by locating each landmark individually using an ensemble of local regression models and appearance cues from selected landmarks. We first detect a set of reference landmarks which were selected based on their saliency during training. For each landmark, an ensemble of regressors is built. From the locations of the detected reference landmarks, each regressor infers a candidate location for that landmark using local geometric constraints, encoded by a point distribution model (PDM). The final location of that point is determined as a weighted linear combination, whose coefficients are learned from the training data, of candidates proposed by its ensemble's component regressors. We use multiple subsets of reference landmarks as explanatory variables for the component regressors to provide varying degrees of locality for the models in each ensemble. This helps our ensemble model to capture a larger range of shape variations as compared to a single PDM. We demonstrate the advantages of our method on the challenging problem of segmenting gene expression images of mouse brain. The overall mean and standard deviation of the Dice coefficient overlap over all 14 anatomical regions and all 100 test images were (88.1 ± 9.5)%.